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ABSTRACT 


The problems of estimation and control of discrete, linear, time- 
varying systems are considered. Previous solutions to these problems in- 

• 9 ■ 

volved either approximate techniques, open-loop control solutions, or 
results which required excessive computation, 

i * 

The estimation problem is solved by two different methods, both 
of which yield the identical algorithm for determining the optimal filter. 
This algorithm is the partitioned equivalent to a solution suggested in 
1964 by formulating the time delay system in an "expanded state" repre¬ 
sentation. The partitioned results achieve a substantial reduction in 
computation time and storage requirements over the expanded solution, 
however. The results reduce to the Kalman filter when no delays are pre¬ 
sent in the system. 

The control problem is also solved by two different methods, 
both of which yield identical algorithms for determining the optimal 

v 

control gains. This result is also a partitioned solution to the "ex¬ 
panded state" representation of time delay systems and also achieves 
savings of computation time and storage requirements. The stochastic 
control is shown to be identical to the deterministic control, thus ex¬ 
tending the sepai*ation principle to time delay systems. The results 
obtained reduce to the familiar optimal control solution when no time 
delays are present in the system. 


• * 

' *■ .* 

" * •*. . • 

* v 

- - *• 

* « > ' • 

• * 2 \* - . \ : .• • + * 9 

* 

♦ 

* 

l 

* 

* 

• x 

* * 

A 

The principle of duality of estimation and control is shown to 
he extended to time delay systems. 

i , 

Necessary and sufficient conditions are developed r for the obser- 

9 

vability and controllability of discrete linear systems with time delay. 

An exhaustive bibliography of publications dealing with optimal 

* 

* 

estimation and control of time delay systems over Lhi period 1960-1970 
is included. 




CHAPTER, l 


INTRODUCTION / 

1.1 Introduction, 

* 

The study of systems with time delay Is not a new one. During 
the past decade (1960-1970) over 200 papers have appeared dealing with 
the optimal control and estimation of linear systems with time delay* 
Closed-form solutions are difficult to obtain because the analytic ex¬ 
pression for a time delay differs from the form used to describe the 

rest of the system. For example, in the frequency domain, the system 

* 

is usually described by a ratio of polynomials in s, but the time M'.ay 
is expressed as the transcendental function, e * . This may be put in 
a common form by expressing e“ Ts as an infinite series in s. Unfortun- 

* f 

ateiy, this results in a system of infinite order with an,infinite num¬ 
ber of poles. Since the exact solution to such a system is computa- 

* 

tionally impossible, the normal procedure is to truncate the approxima¬ 
tion to the time delay. This results in a solution which is suboptimal. 

* 

In the time domain, the describing equations which are ordinary 
linear differential equations when no delay is present, become differen¬ 
tial-difference equations to describe the time-delay effect. The time 
delay may be represented by a differential equation of infinite order. 
This also results in a system of infinite order, however, so a truncated 
series is generally used, resulting again in a suboptimal solution. < 


1 


A, third way q£ describing the system is in discrete time. This 

* 

is naturally suited for time delay representation because the system and 

* * #. 

t&ne delay can be described as difference equations. A major problem 

0 

0 

exists in such, a representation because the resulting system is directly 
proportional to the magnitude of the longest time delay. Once again the 

a 

computation can quickly become excessive. 

i , 

The approach of this paper is further exploration of the discrete¬ 
time representation of systems with time delay*. Substantial savings in 
both computation time and storage requirements are achieved over previous 

discrete-time solutions to systems with time delay. 

♦ * * 

1.2 Previous Work 

An extensive bibliography is presented in Appendix C which con- 

* 

tains a list of more than 200 papers published in the last decade, which 
deal with time delay systems. In general, these pipers fall into three 

categories which limit the practicality of their implementation. 

* * 

1. The majority of the control results are open-loop. This is 
of little value for stochastic systems where plant disturbances occur. 

2. The computational requirements of the proposed solutions are 
too great for practical implementation. 

3. Approximate methods are used, resulting in suboptimal solu¬ 
tions. 

The previous work, pertaining to this dissertation is discussed in detail 
in Chapters 2 and 3. 


3 



1.3 Organization 

• • 

Chapter 2 develops an algorithm, for optimal estimation of dis¬ 
crete linear systems with, time delay using properties of conditional ex¬ 
pectation. This result is shown to reduce to the Kalman filter when no 
time delay is present. 

Chapter 3 develops an algorithm for optimal control of discrete 
linear systems with time delay using dynamic programming. The stochastic 
control is shown to be the same as the deterministic control for time 
delay systems, thus extending the separation principle to such systems. 
Both results are then shown to reduce to the familiar optimal control 

';i * 

solutions when no time delay is present. 

Chapter 4'introduces the "expanded state" form, an alternate re¬ 
presentation of time delay systems in discrete time. This new form is 
then used to obtain results identical to those of Chapters 2 and 3. The 
expanded state representation of time delay systems is not unique to this 
paper. It has been examined by previous 'authors and found to be compu¬ 
tationally unattractive because of the resulting increase in system or- 
• * 

der which is proportional to the magnitude of the longest time delay. 
Chapter 4 results in a partitioned algorithm solution whereas previous 
authors retained the entire expanded matrices in their original form. 

Analytical expressions are developed in Chapter 5 for the com¬ 
putational and storage requirements of the partitioned solutions and 
compared with the requirements of the expanded state representation. 
Although identical results are obtained to the estimation and control 





problems, the partitioned solutions of; this paper result in a 30% to 

90% reduction in computation time, and a 30% to 60% reduction in storage 

* 

.requirements over the expanded form. 

Chapter 6 summarizes the results and suggests future areas of 
study for time delay systems. '.f 




CHAPTER IT. 


OPTIMAL ESTIMATION IN LINEAR DISCRETE SYSTEMS 

WITH. TIME DELAY 

2.1 Introduction 

In this chapter, the* problem of estimation is examined for sto¬ 
chastic linear discrete systems with, time delay. It is assumed that 

B. 

both, plant and measurement noise are present. An algorithm is obtained • 
for estimating the state of the system. 

Section 2.1 formulates the general estimation problem for dis¬ 
crete systems., whether time delay is present or not. The familiar dis¬ 
crete-time system model is, modified to incorporate the effects of de¬ 
layed state values into the system behavior. 

In Section 2.3, previous work, on stochastic time delay systems 
is reviewed. One of these works, that of Priemer and Vacroux,^ obtains 
a similar result to that obtained in Section 2.6. The result in this 
chapter is achieved using properties of conditional expectation whereas 
the referenced work uses orthogonal projection. This result also in¬ 
cludes systems with a control input whereas the work of Priemer and 
Vacroux does not. 

Section 2.4 establishes basic results in estimation theory which' 
are necessary for the results of Sections 2.5 and 2.6. In some instances 
the proofs accompany these basic results. In others where the proofs 
are considered unnecessary or burdensome, the reader is referred to the 
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literature. 

* 

Section 2.5 uses the properties of; conditional expectation to 

* 

develop an 'expression for the one-step prediction process., 

* 

Properties of conditional expectation are again used in Section 

2.6 to develop the estimation algorithm for stochastic discrete-time 

* » * * 

systems with, time delay. This- result is demonstrated to reduce to the 

familiar Kalman filter when no delay* is present in the system. 

Section 2.7 summarizes the results, of this chapter and discusses 
the computational aspects of the estimation algorithm obtained. 

2.2 Optimal Estimation for Discrete Systems 

In this section the general problem of estimation, for discrete 
systems is considered. That i's, the state and measurement processes of 
a dynamic system are discrete-time stochastic processes. 

Consider a dynamic system S whose state as a function of time 
is an n-dimensional discrete—time stochastic process {x(.k) , kel} where 

either I « {k:k - 0, 1, ... , N} or I - {k:k - 0, 1, 2, ... } . Sup- 

• • 

pose that it is desired to know the value of x(k) for some fixed k, 
but that x(k) is not directly accessible for observation. In addition, 
suppose that a sequence of measurements z(.l), ... , z(j) are available 
which are causally related to x(k) by means of some measurement system 
M as shown in Fig. 2.1^and it is desired to utilize these data to infer 
the value of x(k). Let £z(i), i = 1,2,..., j} be an m-dimensional, 
discrete-time stochastic process. 

. Since only the measurements zCl)> •.• , .z(j) are available from 
which to estimate x(k), let the estimate of x(k) based on these 






Figure 2.1 Block Diagram of Dynamic System, S, With State x(k) and 
Measurement System, M, With Measurements z(i) 



measurements, be denoted by x(k| j) and define it to be some n-dimensional 

• * 

vector-valued function f^ of the measurements! 


x(k|j) * f. iz(i), i « 1, ... , j) 


The estimation problem is, one of determining f^ in some rational 

». 

and meaningful manner. The approach, to be used in this chapter is based 


on consideration of the estimation error, x(k|j), which, is defined by 
the relation , 


xCk-U) = x(k) £(k|;j) (2.1) 

Ideally, x(kj j) = 0 and the estimate is exact, When x(k|j) £ 0 

* 

a penalty is assigned for the incorrect estimate. This is done by 
specifying a penalty or loss function L = L{x(k|j)} which has the follow 
ing properties: 

1. L i.s a scalar-valued function of the n variables 

2. L(0) = 0 where 0 denotes the null n-vector 

— .«*> 

3. Ltx b (.k|j)J < L[x a (kJ j )] whenever p[x b (k|j)] <p[x a (k|j)] 
where p is a scalar-valued, non-negative, convex function 
of the n-variables - 

4. Llx(k|j)] = L[-xCk|j)j • ' 

The first property is essential to the obtaining of a unique 
minimum associated with a family of vectors. 

The second property simply specifies that there is no penalty 
when the estimate is exact. . 

In the third property, p is a measure of the distance of x(k| j) 
from the origin in n-dimensional euclidean space, and L is specified to 
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be a, non-decreasing function of this distance. That is, as x(k|j) be- 
comes "closer" to zero, the penalty decreases. 

The*fourth, property requires that L[»J be symmetric about the 

/ 

* * 

origin. 

A loss function that possesses, the above four properties is 

termed an admissible loss function . It should be noted that L need not 

* ♦: 

be a convex function. 

Since x(k) and x(k| j) are random vectors, it follows that x(k|j) 
is also a random vector and that L is a random variable. In order to 
obtain a useful measure of the loss, a performance measure J is defined 
as the mean value of L. 

The familiar discrete-time model for a stochastic process may be 
expressed as 

x(kd*l) = $ (k+l,k)x(k) + ^ (k+l,k)u(k). + p (k+l,k)w(k) (2.5) 

z(k+l) = H(k+l)x(k-KL) + v(k+l) (2.6) 

where x = n vector (state) 

u - r vector (control) . ~ 

w = p vector (plant disturbance) 
z - m vector (measurement) 
v -• m vector (measurement disturbance) 

$ => n x n (state transition matrix) 

p - n x p (disturbance transition matrix) • 

. f = n x r (control transition matrix) , 

H. = m x n (measurement matrix) 



t 


•4 

,1 




I 'T~% 

i 


s 

LJ 
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J 

(w(k), k « 0, 1, ...} * zero mean gaussian white sequence having a posi- 

¥ 

tive semi-definite p x p covariance matrix {Q(k), k B 0, 1, ... }. That 
is,. 

/ 

* • 

*% 

fc(w(k)} « 0, k « 0, 1, ... . (2.7) 

fclwCjVCk)} - QCk)« . k (2.8) 

' . 

where is. the Kronecker delta. 

, fc(kfl), k=0,1,...} is a zero mean gaussian white sequence having a posi 
tive semi-definite m x m covariance matrix (R(k-KL) , k=0,l,...}. 




m 

1* 


0 


i 

WM 




fe£v(k+l)> = 0, k = 0, 1, ... 


E{vCj+1)v' (k+i)} - RCk+l)^ jk 


C2.9) 

(, 2 . 10 ) 


Attention i.s restricted to the case where the two stochastic processes 
I'iCk), k = 0, 1, ...} and‘{v(k+l), k- 0, 1, *..} (2.11) 

are independent of each other. That is 

E jv (j)w'(k)} = 0 for all j = 1, 2, ... , k = 0, 1, ... (2.12) 

The initial state x(0,) is a gaussian random n-vector with zero mean, 
having an n x n positive semi-definite covariance matrix 


S >:! 


EixCO)x’(O)} = P(0) 


(2.13) 


m 


ti¬ 


lt is assumed that x(0) is independent of (w(k) , k « 0, 1, ...} and 
’ {v(k+l), k « 0, 1, ... } so that 


E£c(jO)w’(k)} = 0, k - 0, 1, ... 


(2.14) 



I 

i 
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E{xCO)v’(k+1)} « 0, k B 0, 1, ... (2,15) 

a 

It is also assumed that the cdntrol sequence Is either known or may be 
specified. 

Expression (2.5) states that the value of the state vector at 
the* next Instant of time depends only on the present values of the 
state, control, and plant noise vectors. If, however, this future value 
of the state vector is also dependent upon past values of the state vec- 
tor^ then Eq. (2.5) must be modified to indicate this dependence. This 
is done by re-writing Eq. (.2.5) as 

J 

, x(k+l) =• t £ ({, i (k.-M,k)x(k,-i) + ij;(k-+l,k)u(k) + f(k+l,k)w(k) (2.16) 

The index J in Eq. (2.16) indicates the most distant (in time) value of 
state vector, x(k-J), that affects the state vector at the next instant 
in time. I.t is recognized, of course, that many of the <J>^(k+l,k) may 

be zero, indicating that not all delayed states back to x(k-J) affect 

* • 

the expression for x(,k+l). Thus <j>^(k+l,k) is the state transition matrix 
associated with the delayed state vector x(k-i) in Eq. (2.16). Proper¬ 
ties expressed by Eq. (2.13), (2.14) and (2.15) must also be modified 
accordingly, as below, to include systems with time delay. 

The initial and delayed initial state (x(-i), i - 0, 1, ... , J) 
is a gaussian random n-vector with mean 

E {x (—i) } - 0, i “ 0,1,. . ., J 

having the n x n positive semidefinite covariance matrices 


(2.17) 



E{x(-i)x'(-j)} - P(-i, -j) i, j « 0, 1.J. 


(2.18) 


It is assumed that' £x(-*i), i « 0,1,is independent of 

»* 

(w(k), k » 0, 1, ...) and £v(k+l), k * 0, 1, ...} so that' 


E{xC-i)w'(k)} = 0 i«0,l,...,J k « 0, 1, .. 


(2.19) , 


Efxt-iW (k+1)} * 0 - i « 0,1,..., J k « 0, 1, ... (2.20) 


In addition, it should be noted that the system described by 

Eq. C2.16) and Eq. (2.6) must be observable or the following theory has 

# 

little meaning. A system is "observable" if every state in the corres¬ 
ponding deterministic system can be exactly determined from measurements 

* 

of the output over a finite interval of time. The reader is referred 
to Appendix A for a discussion of observability of discrete linear sys- 
terns with. time, delay. 

With the system model described by Eqs. (2.6) - (2.11) and Eqs. 
(2.16) — (2=20), the specific estimation problem may now be formulated. 

. Given the state sequence &(k), k - 0, 1, ...} and the measure¬ 


ment sequence fe(i), i - 1, 2, ... , determine the estimate, x(k|j), 
of x(k) such that the expected value of- the mean square filtering error 
is. minimized. That is, it is desired to minimize E (L[x(k| j) ] ) where 


II X (k I j ) J = x.(k! j) X ' (jk|j; 


( 2 . 21 ) 


x(k | j ) = x(k) - x (k j j ) 


( 2 . 22 ) 


c 
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. The remainder of this chapter is devoted to determining the op- 

♦ 

tiraal filtered estimate {x(k|j), tevj). As a byproduct, however, e*- 

pressions are also obtained for the one-step prediction estimate 

* * 

ixCKIj)» k“j+l) and for a limited range of smoothing estimates 
£S<k|j), k-j-1, i-2, ... , j-j}. 

* 

These estimates are all obtained for discrete linear systems 

» 8 

with, time delays. 

2.3 . Review*of Previous Work on Estimation of Time Delay Systems 

Despite the great number of papers 1 extending the original work 

#. 

2 

in 1960 of Kalman and Bucy on linear filtering, a number of years 
passed by before optimal filtering of systems with, time delays was dis¬ 
cussed. The bibliography contains an extensive list of such literature. 
The three articles which discuss linear systems with delay in the plant 
only (not in measurement or control) are discussed below. 

♦ 

In 1967, the theory developed by Kalman and Bucy was first ex- 
tended to linear systems with, multiple time delays by Kwakernaak whose 
development is for continuous systems. The single-variance equation of 

t 

the Kalman-Bucy theory is replaced by a partial differential equation 
and three boundary conditions. The boundary conditions are also partial 
differential equations. No explicit solution to these equations appears 
possible in closed form and solutions are not feasible for on-line anal¬ 
ysis of discrete time systems. Thus, although the solution is theoreti¬ 
cally presented, i.t is impossible to implement in practice due to the 
extensive computation required. Thi.s - is substantiated by the fact that 
no literature appears in which this method is implemented. 


The problem of developing a filter that was computationally 

* 

* 

more feasible and that could be used in practice was resolved in 1969 

by Priemer and Vacroux.^ They offered a solution for discrete linear 

* 

t 

* * 

systems with, time delay which, avoided the necessity of expanding the 

order of the system. The computation time was considerably reduced 

* 

3 * 

from that required by the method of Kwakernaak. Chapter 4 of 

this dissertation demonstrates that Priemer’ and Vacroux's result is iden¬ 
tical to that obtained by expanding the state space, 

t 

In addition to these papers is the method of approximating the • 
time delay by a Taylor's Series expansion or by a Pade approximation. 
Unfortunately, the solution obtained is suboptimal^since a truncated 
approximation to the delay is used. No work appears in the literature 

discussing this type of approximation in relation to filtering of time 

* 

delay systems,. 

' » 

Both, papers discussed above consider the filter problem only 


and do not provide a complete solution for optimal control of stochastic 

* 

time-delay systems. In 1969, however, Larson and Wells^ were able to 
do so by restricting attention to serial systems . Although their solu¬ 
tion was optimal only for single-input, single-output systems with delay 

in the plant, it is very easy to implement and computationally 

faster than any of the methods discussed in the above paragraphs. Their 

paper combines optimum estimation, prediction and control. 

It is also recognized that if it can be shown that the separation 
principle applies to time delay systems ; then the optimal filter may be 

developed independent of the optimal control solution. This is, in fact, 

... * . 

demonstrated in Chapter 4 of this dissertation. 


■ % - 
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In summary, them, with one exception, the results shown thus far 
In the literature for estimating systems with, delays in the plant, are 
limited, as to practical application., by one or more of the following: 

1. Excessive computation time, 

2. Excessive computer storage requirements t 

3. Approximate methods resulting in suboptlmal results 

The. one exception i.s the paper by Pri.emer and Vacroux. Their results 
are obtained by tw.o alternate methods in this dissertation. Conditional 
expectation properties are used in this chapter and an expanded state- 
representation is used in Chapter A. In both, cases, the work of Priemer 
and Vacroux i.s extended to include control inputs, 

2.A . Fundamental Theory of Estimation 

In this section some fundamental results of discrete estimation 
theory are obtained. Although., in some cases, more general results may 
be established, only those which are necessary- to the development of 
Sections 2.5 and 2.6 are presented. The reader is referred- to the 
literature in those cases, where the presentation of the proof is felt 
to be burdensome .or unnecessary. 

The following properties of gaussian conditional expectation are 
of fundamental importance in the next two sections. Here x, y, z are 

5 

gaussian random vectors. For proofs the reader is referred to Meditch 
(pp. 92-103) . 

1. Ebc|y} is a gaussian random vector which is a linear com¬ 
bination of the elements of y. 


(2.23) 
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2. x - £{x| y} i.s independent of the. random vector ohtained by 

any linear transformation on y. (2.24) 

3. If y and z are independent^ where z i,& a rand ora’ m-vec tor then 

* * 

E{x|y,z} «E{x|y) + E{x| z) -x (2.25) 

4. For y and z not necessarily independent, 

* * * 

E(x|y,z} « E(x|y,z} - E^x|y) + E(x|z) - x (2.26) 

where t -■ z — E{zjy). 

* 

The following theorem can now be proved. 


Theorem 2.1 . If the-loss function is defined as 
L *= [x(.k) - R(k| j)J * [x(k) -£(k|j)J and 
{x(k), kcl} and’ (z(i), i - 1, ... , j} are dis¬ 
crete time stochastic process.es. then 

£(k.|j) - E {x(k) | z*(j) } (2.27) 

where z*(j) m z(0) 

z(l) 


,(j)j 

Proof: Recall from the problem statement, Eq. (2.21) and Eq. (2.22) , 
that x(k|j) is to be chosen such that L is minimized. By taking the 
gradient of L in Eq. (2.21) with respect to x and setting it equal to 
zero 


V L = -2EKk) - x(k|j) \z(l) , 


, z(j)} - 0 


• • 


(2.28) 



Taking the transpose of hoth, aides of Eq. (2.28) 


E{x(.k) |z(l), ... , z(j)} « E{x(k| j) |z(l) .. , z(j)} (2.29) 


But the right-hand side of Eq. (2.29) la just 

Eft(k|j)|z*(j>> = ACk|j) 


(2.30) 


By substituting this result into’Eq. (2.29) 


£ (fc | j ) ** E{x(k)|z(l), ... , z(4)} 


(2.31) 


and the theorem is, proved.- 

* * 

Now: the final four properties necessary to the development of 

this chapter can be stated. The reader is again referred to Meditch** 
(pp. 92-103) for proofs. 

5. x(k,|j) and x(kjj) are gaussian random n-vectors 

m 

Where x(k| j) =• x(k) - £ (k | j ) . (2.32) 

6. x(k|j) is. independent of any linear combination of the 

available measurements. • (2.33) 


7. *(k|j) is unique. 

8. E{x|y}> x + P^ y ,Pyy.(y-y) 


(2.34) 

(2.35) 


where P ** E {xy *} 

. xy 

P « E{yy'} 

yy 

x ~ E{x) 
y - E{y} 


This concludes the results, -necessary to develop the optimal 


filter. 




2.5 . Optimal Prediction for Discrete Linear Systems with Time Delay 

«► t 

Although, the results of the preceding section are fundamental 
ones, they are of limited practical utility. Consider the gaussian case 
where the optimal estimate is given by Eq. (2.35) as 

*(k|j) » x(k) + P x ( k ) z *(j)i’zA(4)z*Cj) lz * (j) ■ ** ( nJ < 2 - 36 ) 

where z*(j) * z(.0X 

2 ( 1 ) 


2 (j ) 


For each set of measurements, it is necessary- to compute the inverse of 

P a jm x jm matrix, where j is the number of measurements and 

z*(j)z«(j) 

m is the number ‘of elements,in the measurement vector. If m is 1 and 
there are 50 measurements, then a 50 x 50 matrix must be inverted. If 
it is desired to perform estimation "on-line", application of the above 
expression to generate the optimal estimate becomes impractical. 

What are desired, from a computational point of view, are 
efficient and practical algorithms for-processing the measurements se¬ 
quentially, hopefully in real time, to obtain a current estimate. All 
of the results stated thus far are independent of the system model. For 
this reason they are valid for the time delay case also. The remainder 
of this chapter is devoted to developing such algorithms for prediction 
and estimation of the states of a discrete linear system with time delay 


,4 
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2.5.1.. System Model Properties 

1 - " n ^ ■ ■ ■ 1 • 1 1 11 r ' 1 ■ - 1 nr: n ■ i ■■ -t ^ 

The system model is described by Eqv (.2.6) - (2.11) and Eq.' 
(2.16) -> (.2.20) where the fundamental system and measurement equations 


are 

J 

x(Jc+l) = £ <K (k+l,k)x(kKi.) + r(k+l,k)w(k)+i|j(.k+l,k)u(k) (2.37) 

* » 

z(k+l) = H.(k+l)x(k+l) + v(k+l) (2.38) 


This model has, the following properties,. Proofs are in Appendix B,. 

1. The stochastic processes, (x(k), k = 0, 1, ... } and 

lz(k), k ** 1, ... , j) are gaussian, (2.39) 

2. Elx(j)w' (Jk)} « 0 for all k > j, j - 0, 1, 2, ... (2.40) 

3. E {z(j)w' (k)} « 0 for all k„ £ j,’ j =0, 1, 2, ... (2.41) 

4 f E{x(j)v f (k)} = 0 for all j and k. (2.42) 

5. E{z(j)v , (k)} = 0 for all k > j .• (2.43) 

I 

.* 1 ! 

' i: 

It is helpful in the sequel to define the following error 

. ii . 

a 

covariance matrices in terms of 


P(i,m|k) = E{[x(i) - £(f |k.)] [x(m) - ^(mlk)]'} (2.44) 

Case 1^ A, m>k; prediction Error Covariance Matrix (2.45) 
Case 2: £>k, m = k; Prediction/Filtering Error Covariance Matrix (2.46) 
Case 3/. i>k, m<k; Prediction/Smoothing Error Covariance Matrix (2.47) 
Case 4_: A = m = k; Filtering Error Covariance Matrix (2.48) 
Case .5: A = k, m-<k; Filtering/Sraoothing Error Covariance Matrix (2.49) 
Case 6: A, m^c; Smoothing Error Covariance Matrix (2.50) 



It should als,o be clear from Ec^. (2.44) that 


P(£,m|k) P 1 (in, & | k) (2.51) 

where x(.Jl jk) *=* x(£) -> x(Jl|k) / 

■* prediction, error if l > k, (2,52) 

** filtering error if l » k (2.53) 

*= smoothing error if % < k. * (2.54) 


2.5.2 . Optimal One-Step Prediction * • • 

The algorithm for the optimal one-step predicted estimate 
x(k+l|k) is developed below. Some important properties of the corres¬ 
ponding one-step prediction error x(k+ljk) = x(k+l) - x(k+l|k) are also 

»- 

established. Of particular interest is the nature of the stochastic 
process x(k-KL|k), k - 0, 1, ... and the behavior of its corresponding 
covariance matrix 

. P(k.+l,k+l|k) - E {x(k+11k)x' (k+11k)} * (2.55) 

It is assumed that the optimum estimates {x(k-i|k), i=0,l,...j} 
and the n x n covariance matrices {P(k-i,k-j|k), i,j = 0, 1, ... , J} 
of the corresponding filtering and smoothing errors x(k-i) and x(k-j) 
are known for k = 0, 1, ... ; i,j - 0, 1, ... , J. The procedures for 
obtaining x(k-ijk) and P(k-i,k-j|k.) are given in the next section. 

Prom the property given by Eq. (2.39) and Theorem 2.1 
*(j-i|j) - E £x(j-i) | z (1) , ... , z(j) } (2.56) 

is the optimal estimate of x(j-i) for j = 1, 2, ... ; 1 = 0,1, ... , J. 
For j - 0,'there are no measurements and it follows from Theorem 2.1 that 


jeer'll 0 ) ** E {x (-i) | no measurement a) 

* • * 

■ E {x (rl)} 

* 

-0 . ( 2 . 57 ) 

/* 

.# 

It ts clear that i(j~i|j) Is gaussian from property (2.32). 

Similarly the filtering error ;j). =* x(j) ~ £(j-i|j) is a zero mean 

gaussian random n-vector for which, it is assumed the covariance matrix 

j) is given. For j = 0 , 

• * 

s(-i|0) = x(.-i) - S(-i|0) 

t 

* xC-i) - 0 ( 2 . 58 ) 

* 

so that P(ri.»-i|0) - E{x(.-i| 0)x’ (-&|0) 

« E (x(ri)x’C h) 

« PC-i,-i) i, l * 0,1,.. .,J (2.59) 

where, the latter is assumed given In the system description. 

The following result can now be established for optimal pre- 

diction. 

Theorem 2.2 If the optimal filtered estimate 
£(j-i|j) and the covariance matrix P(j~i, j-iij) 
of the corresponding filtering error 
*(j~i|a) * x(j-i) - 2Cj-i|;j) are known for some 
j = 0, 1, . ; i> i. - 0, l s • •. , J, then for 

k = j + 1 

(a) The optimal predicted estimate &(k-KL[k) 
for all.admissible loss functions is given by the 
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express ipn 

J 

x(k+l|k) * ,£ $ i (k+l,k)x(k~i| k) 

i*?0 


+ \p (k+l,k)u(k) (2.60) 

(b) The stochastic process {x0t-+l| k), 

* 

k « 0, 1, 2, defined by the prediction 

* 

error relation 

0 . I. 

x (k-f 11 k) =. x(k+l) - x(k+l|k) 

i 

is a zero mean Gauss Markov-CJ+l) sequence whose 
covariance matrices are governed by relations 


J J 

P(k+l,k+l|k) « E E (f j (k+1,k)P(lc-1,k-j| k) $! (k+1,k) 

1=0 j =0 J 


+ r(k+l,k)Q(k)r'(k+l,k) 


Proof ; From Theorem 2.1 and Eq. (2.16) 


x(jk+l|k) = E {x(k+l) | z(l) , ... , z(k)} 


J 

x(k+l) = I <J>.(k+l,k)x(k-i) + r(k+l,k)w(k) + Kk+l,k)u(k) 
i-0 


The substitution of Eq. (2.16) into Eq. (2.62) yields 


(2.61) 


(2.62) 


(2.16) 


J 

Jt(k+l|k) * E{ t $.(k+l,k)xCk-i) + f (kU,k)w(k) 
i«0 X 

+ i/>(k+l,k)u(k)| z(l).z(k)} 

» * 

$ 

/ 

J 

“ £ $.(k+l,k)E{x(k-i)|z(l), ... , z(k)} 
t“0 x 

1 4 a 

+ rCk+l > k)E{w(k)|za>, ... , z(k)} 
+ ,j,(k+l,k)fc{u(k)|z(l), ... , z(k)} 


(2.63) 


The application of property. ,(2.41) and Eq. (2.7) to the second term 
causes It to vanish. 

E fw(k) | z(1), ... , z(k)} « E{w(k)} - 0 (2.64) 


Under the assumption that the control sequence {u(k), k-0, 1, ... } is 

known or can. be specified as desired, the third term in Eq. (2.63) be-- 

* 

comes 

* ♦ 

(k.+l,k.)E£u (k) |z(1) , ... , z(k)}. = «(k.+l,k)u(k) £2.65) 

* 

The substitution of Eq. (2.64) and Eq. (2.60) reduces Eq. (2.63) to 

* / 

J 

*(k+l|k) = £ <f>. (k+1,k)&(k-i|k) + (k+1, k) u (k) (2.66) 

i-0 1 . 

which verifies Eq. (2.60) of Theorem. 2.2. from, the definition of pre¬ 
diction error (2.52), filtering error (2.53) and smoothing error (2.54), 
application of Eq. (2.\ 16) and Eq. (2.66) yields 
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x (k+1 1 k) * x(k+l) - £ (k+1 J k) 

, J 

» t <P j (k+ 1 ,k)x(k-i) + r(k+l,k)wCk) + ip (k+l,k)u(k) 
’ i^O 


— £ <f>.| (k+l,k)k(k-ij k) - $ (k+l,k)u(k) 
i«0 . 

J 

<Mk+l,k)x(k-i|k) + ^(k+l l k)w(k) C2.67) 

1 * 0 . * * 

It remains to establish, that x(k+l|k) 'Is a zero mean Gauss-Markov 

t • f 

•^(j+1) sequence. This can be done by examining Eq„ (2.67). Since w(k) 

is gaussian and £x(.-i|o), 1 « 0, 1.j} is gaussian, it follows 

that x(k+l|k) is a zero-mean discrete-time gaussian sequence. The 
Harkov property also follows immediately from Eq. (2.67). In fact tills 
has a Markoy -(J+1) property since the sequence depends on events occur¬ 
ring (J+1) time intervals in the past. 

.. ; » 

Now the expression may be determined for the prediction error 

■ * t 

covariance matrix. 


P(k+1 1 k+11 k) * E{x(k+l|k)x , (k+l|k)} 

J 

* kX[£ <j> (k+l,k)x(k-ijk) + f(k+l,k)w(k)] 
i*0 * 


• [ t <J> . (k+1 ,k)x (kr j I k) + r<k+l,k)w(k)]'} 

a —n J 


J J 

** l I <J> a (k+1, k) E {x (k-i| k)x 1 (k-j j k)}<•) f . (k+1, k) 
i=0 j=0 J 

J 

+ 2Z <p. (k+1, k) E { x (k- i | k)w’ (k)}T 1 (k+l,k) 
i-0 * 

+ r (k+l,k)E{w (k)w* (k))r ’(k+l,k)} C2.6S) 


From the definition of filtering error, Eq. (2.53), and smoothing 

* 

error, Eq. (2.54), 

E{xCk.-i|k)v? t (k)} « E{[x(k-i) - x(k-i|k)J [w* (k)]} •' 

«* E{x(k~i)v?’ (k)} - E &(k~i|k)w* (k)} (2.69) 

The first term of Eq. (2.69) is identically- zero due to property (2.40). 
Further, since x(k-i|k) is a linear combination of the measurements it 

m 

may be expressed as 

k 

x(k~i|k) * i A(i)z(*) (2.70) 

i**i * . 

' ' ¥■ 

The substitution of Eq. (2.70) in the second term of Eq. (2.69) causes 
it to vanish, also* 

* 

. k 

E{x(k-i|k)w’ (k)} ** E( £ A(i)z(f)w t (k)) 

£*<L 

* 

k 

** l A(i)E {zCi )^ 1 (k) } 

A~1 

« 0 (2.71) 

The application of property (2.41) causes the cross-product terms of 
Eq. (2.68) to vanish. Thus Eq. (2.68) becomes 

J J 

PCk+l,k+l|k) = £ Z $ . (kH-1,k)E(x(k-1| k) x 1 (k-j | k) } 4' (k+11 k) 

• i-0 j=0 x J 

* 

+ 1?(k+1 ,k)E {w(k)v' (k) }r’ (k+1,k) 


The substitution of Eq. (2,44) and Eq, (2.8) gives 

J J 

P(k+l,k+l|k) * £ 1 <{, .(k+l,k)ECk~l,k^|lc)(|»’(k41,k) 

1*0 j «0 1 J 

* 

+ r(k+l,k)Q(k)r i (k+l,k) (2.72) 

'* 

Note, ?,at this point, that the solution to the single-stage opti- 

4 . i 

* 

mal prediction problem is solved, The associated error covariance matrix 
expressed by Eq. (2.72) requires knowledge of P(k-i,k-j | k), however, and 

• 

this is known only for k « 0. In the next section this problem is re- 
'solved and expressions for the filtering-and smoothing error covariance 

matrices P(k-'l,k-^| k) are obtained. This information is then combined 

* ». 

with that of the single-stage optimal predictor to obtain the optimal 
filter for time-delay systems. 

2.6 . Optimal Filtering for Discrete Linear Systems With Time Delay 

In developing the algorithm for optimal filtering for the sys¬ 
tem of Eq. (2.37) and Eq. (2.38) it is assumed that only the initial 

estimate {k(-i|0) - 0, i-0,1.J}, the covariance matrices of the 

filtering and smoothing errors at the initial time P(-i,-j|0) - 
E{x(-i|0)x‘(-j|0)) - E{x(-i)x(-j)} = P(-i,-j) and the set of measure¬ 
ments {z(1), ... , z(k), z(k+1), k > 0} are given. 

From Theorem’2.1 the optimal filtered (i-0) and smoothed 
(i—1, •. * , J ) estimates £(k-KL-i| k.+l) are given by the relation 

£(.k+l-i|k+l) - E {x (k+l-i) J z (1).z(k), z(k+l)} i=0,l,...,J 



=0*** - 



Property (2.26) may now, be applied to examine (2.73). 

♦ 

* * 
w 

fc{xj)r,z} « £{xj y) + fc{x|z) - x “ (2.26) 

* 

. / 

*■ 

* 

where z z - E{z| yl. 

By substituting (2.26) into (2.73) and noting that x « \|>(k+l,k)u(k) 

* ... . 

% 

fc(k+l-i| k+1) = E{x(k+l-i)| z(l), ...» z(k)> 

+ E{x(k+l-i)| z(k+i|k)} - i> (k+l,k)u(k) (2.74) 

for k 0, 1, ... and where 

•.* * 

* S (k+11 k) « z (k+1) - z (k+11 k) 

* z(k+l) - E{z(k+l)|z(l)., , z(k)} (2.76) 

This difference z(k+l|k) is called the measurement residual . By sub- 

>. * 

stituting Eq. (2.38) and solving for z(k+l|k) 

z (k+11 k) = E (H (k+1) x (k+1) + v(k+l)| z(l) , ... , z(k)> 

1 *. 

* H.(k+1) E {x (k+1) j z (1) , ... , z (k) } /! 

+ E{v(k+1) | z(l) , ... , z(k)) 

« R(k+l)x.(k+l|k) + E(v(k+1) | z (1) , ... , z(k)) (2.77) 

The second term vanishes by Eq. (2.43) and Eq. (2.77) becomes 

2(k+1|k) = H(k+l)x(k+l|k) > k - 0, 1, ... (2.78) 

Uith these preliminaries completed, the basic theorem for optimal filter 
ing of disdf#>te linear systems with time delay may now be proved. 
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Theorem 2.3 . (a) The optimal filtered estimate 

Sc(k+l| k+1) for the system described Oy Eqs. (2,37) and 
(2.38) is given by the recursive relation 

jt 

x (k+11 k+1) = x(k+ljk) + K^Oc+l) [z (k+1) - x(k+ljk)‘] (2.79) 

which is a specific case of the more general expression 
*(k+l-i|k+l) = x(k+l-i| k) + (k+1)fz(k+1) - x(k+l|k)] (2.80) 

for k = 0, 1, ... where x(-i|0) = 0, i = 0, 1, ... , J. 

(b) K^(k+1) is an n x ra matrix which is speci¬ 
fied by the set of relations 

K^Ot+l) = P (k+l-i,k+11 k)H. 1 (k+1) IR(k+1)P (k+1,k.+l | k)H 1 (k+1) 

+ R(k+l)J~ 1 i = 0, 1.J (2.81) 

J 

p(k.-j,k+l|k) = z P(k-i,k-j|kH:(k+l,k) j = 0,1.J“1 (2.82) 

1=0 X 

P(k-i,k-j|k) = P(k-i,k-j |k-1) - K 1 (k)j(k)P(k,k-j|k-l) (2.83) 

" 0, 1, ... , J 

J J 

P(k+1,k+1 Ik) ~ z z <|>. (k+1,k)P(k-i,k—j Ik)<j > x (k+1,k) 

1=0 j =0 1 

+ X(k+l,k)QCk)p*Ck+l,k) (2.84) 

for k = 0, 1, ... and P(-i,-j |0) = P(-i,-j) is the ini¬ 
tial condition for Eq. (2.84)= 

(c) The stochastic process 

{x (k+11 k+1), k = 0, 1, ...) which. is defined by 
x(k+l|k+1) =‘x(k+1) - x(k+l|k+1), k = 0, 1, ... 
is a zero mean Gauss-l-Iarkov -(J+l) sequence whose co- 
variance matrix is. given by Eq. (2.83) for i = j =0. 



1 


*■;. *■ • 

* • 


&- ' * -• v -• • 2 *~ «*• - .; 


Proof : From Eq. (2.74) 

x(k+l-i|k+l) « x(k+l-i|k) + E{x(k+l-i) (z(k+11k) } - ^(k+l,k)u(k) 

(2.85) 

» * t * 

Further, since x(k+l) and z(k+l|k) are gaussian, property’ (2.35) may be 


invoked 


Elx|z} « x + P ~P~~z 
1 xz zz 


(2.35) 


to obtain the following result 


*(k+l-i|k+l) « Tj>(k+l,k)u(k) + £ (k+l-i jk) + E &(k+l~i) z 1 (k+11k) } 

• [E {z(k+1| k) z' (k+11k)} J*z(k+11k) - ip (k+1 ,k)u(k) 

( 2 . 86 ) 


By defining K^(k+1) s E{x(k+l-i)z 1 (k+11k))[E (z (k+l| k)z’(k+11 k))] 

i - 0, 1, ... , J 
Eq. (2.86) may be rewritten 

x (k+l-i | k+1) = x (k+l-i | k) + K 1 (k+l)z(k+l|k) 

However, it is clear from Eq. (2.75) and Eq. (2.78) that 1 
z (k+11 k) = z (k+1) - z (k+11 k) 

==■ z(k+l) - R(k+l)x(k+llk) 

The substitution of Eq. (2.89) into Eq. (2.88) yields 


(2.87) 


( 2 . 88 ) 


(2.89) 


£ (k+l-i | k+1) = & (k+l-i | k) + K. (k+1) [z (k+1) - H.(k+l)£(k+l|k) ] 

(2.90) 

which is the result postulated in .Eq. (2.75). The appropriate initial 
conditions are obviously {&(-i{0) =0, i - 0, 1, ... , J). This result, 
Eq. (2.90), combined with that of th.e one-step predictor, Eq. (2.66), is 
sufficient to describe the structure of the filter which is shown in 
Figure 2.2. 


= t <P ± (k+1,k)£(k-iJk) + Kk+l,k)u(k) 
. i-0 


( 2 . 66 ) 
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Evaluation of K. (k+1) From the definition of prediction error and Eq. 
(2.8 9.) 

fc(k+l| k) * z(k+l) 7 &(k+l| k) 

« z(k+l) - R(k+1)x(k+11 k) 

» •% * i 

« R(k+l)x(k+l) + vCk+1) - R(k+1) St (k+11 k) 

* ? « R(k+l)x(k+l|k) + v(k+l) (2.91) 

Consequently, 

*I<fcH>«<fcH> VElz(k4l|k)VCk+l|k>} . 

i E1 I R(k+1)x(k+11k) + vCk-+l)][x , Ck+l|k)H’(k+l) 

+v , (k+l)]} (2.92) 

« H(k+l)E(xOc+l|k)x l (k+1 j k) }R ! (k+1) 

; ’ , + R(k+1) E {x (k+11 k) v 1 (k+1) } 

+ EfvCk+Dx- 1 (k+l|k) }H* (k+1) 

+ E (v(k+1) v' (k+1) > (2.93) 

The middle two terms of Eq. (2.93) are now shown to vanish.. Since one 
is just the transpose of the other it is sufficient to examine only one. 

E{x(k+1jk)v 1 (k+1)} = E(x(k+1)v' (k+1)> - E{x(k+l|k)v'(k+1)} (2.94) 
From Eq. (2.43) the first term of Eq. (2.94) vanishes. From Eq. (2.23) 
and Eq. (2.27) the estimate is a linear combination of the measurements 

k 

k (k-i+11k) = 2, A.OOz(l) i^.O, 1, ... , J (2.95) 

' 1=1 

The substitution of Eq. (2.95) in the second term of Eq. (2.94) yields 

k 

9 E ft (k+11 k) v 1 (k+1)} = E { 2 A o a)z(t)v' (k+1) } 

k 

« t A q OOE{ z(l)v* (k+1)} « 0 (2.95) 

SL =1 


I 
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by the property expressed in Eq. (2.43). Therefore, Eq. (2.93) may be 

* 

rewritten, using Eq. (2.10), 

' P z( k+ I)z( k+ 1) - H(.k+l)p(k+l,k+l| k)lL' (k+1) + R(k+1).' (2.97) 

Evaluation of P ~ in Eq. (2.35) and substitution of Eq. (2.89) yields 

xz 

P x2 *= fi{x(k.+l~i)^ 1 (k+lj k)) • 

« EOx(k+l-i| k) + x(k+l-i|k)]rx , (k.fl|k)H. , (k+l) + v'(k+l)j} 

« E(x(k+l“i| k)x' (k+l| k)}R' (k+1) 

* . + E{x(k+l-i|k)v’ (k+1)} 

+ E lx (k+l-'i| k) x * (k+1| k) } R 1 (k+1) 

+ E{*(k+l~i| k)v* (k+1)}’ (2.98) 

The second term in Eq. (2.98) vanishes due to Eqs. (2.42), (2.96) and 

(2.58). By property (2.33) and Eq. (2.70) the third term vanishes. The 

* * 

fourth term in Eq. (2.98) vanishes due to Eq. (2.96). Therefore, Eq. 

(2.98) becomes . ( 

(k+l-i) z (k+11 k) “ ® ^ (k+l~i| k)^ Ck+11 k) } R (k+1) 

« P{k+l-i,k+1jk}R l (k+1) (2.99) 

The substitution of Eq. (2.97) and Eq. (2.99) into the defining relation, 
Eq. (2.87), for K. (k+1) gives 

K^k+1) - (k+l**i) z (k+11 k)^*z (k+l)z (k+1) 

P(k+l-i, k+11k)H 1 (k+1)[R (k+1)P(k+1,k+11k)H f (k+1) 

+ R(k+1)3“ 1 (2.100) 

for k = 0, 1, ... and Eq. (2.109) of Theorem 2.3 is proved. Note that 
if R(k+1) is assumed to be positive definite, it follows that the re¬ 
quired inverse always exists since P(k+l,k+l|k) is positive semi- 


definite. 
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Evaluation, of P(k+l-i,k+ll k) 

From Eq. (2,72) the expression for P(k+l,k+l| k) is known to be 

J j 

P (k+l, lc+ll k) . j £ 4 , (k+l,k)P(k-t,t-j Ik)*!(k+l,k) 

i-=0 j =0 1 3 

+ r(k+l,k)QCk)r’(k+l,k) (2.72) 

It remains to determine the error covariance matrices 

V (k+l-i,k+l|k) and P(k-i,k-j |k). For i - 0, P(k+l,k+l|k) is defined by 

» 

Eq. (2.72) so the cases of interest are'for i = 1, ... * J. Now to de- 

i 

termine p(k+l-i,k+l|k) for i ~ 1, ... , J. From Eq. (2.37) and Eq. , 
( 2 . 66 ) 

‘ X(k+l|k) ** x(k+l) - x(k+l|k) 

J 

* E ♦ i (k+l, k) x (k~i) + a (k+l, k) w (k) + t|> (k+l, k) u (k) 

1=0 

i- J 

- E 4>..(k+l,k)£(k-i|k) - ij,(k+l,k)u(k) 

1=0 * 

J . 

** E <(> i (k+i > k)i(k-i|k) + r(k+i,k)w(k) (2.101) 

1=0 

From the definition of error covariance matrix and Eq. (2.101) 

P(k-j,k+l|k) = E(x(k-j J k)x 1 (k+l| k) 

I J=01 •.«»J—1J 

J 

= E lx (k-j | k) [w’ (k)r ’ (k+l, k) + l x' (k-i| k)4> ’ (k+l, k) 3} 

1=0 

= E(x (k-j | k)w' (k)} r * (k+l ,k) 

J 

+ E E{x(k-j| k)x* (k-i| k)}(J>! (k+l,k) (2.102) 

1=0 x 

The first term in Eq. (2.102) vanishes due to Eq.. (2.40), Eq. (2.95) and 
Eq. (2.41). Therefore, Eq. (2.102) becomes 
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J 

P(k->j,k+l|k) « £ P(k-j,k-i|k)$’(k+l,k) j * 0, 

t^O *“ 

wliich. completes the proof of Eq. (.2.84) of Theorem 2.3. 

Recall that for i c 0, 

P(k4-l~i,k+l|k) «= P(k+l,k+l|k) 

which is already knovzn from Eq. (2.72). 

* 

* * * » 

Evaluation of P(k^i,k-.llk) 

.Finally, it remains to determine, the error covariance matrix 

* 

P(k-<,k-j |k) « E {x(k-i|k)x ’ (k-j |k) } (2.104) 

*■ * » • 

From the definition of estimation error and substitution of Eq. (2.90) 
and Eq. (2.38) 

x (k-i | k) » x(k-i) x(k-i|k) 

« x(k-i)- [S(k-i|k-l) + K 1 (k){z(k) - H(k)£(k|k-1)}] 

. ** x(k-i|k-l) - K^(k) [E(k)x(k) + v(k) - E(k)&(k|k-1) 3 
« x(k-i|k-l) ,K 1 (k)k(k)S(k|k-l) - Ki(k)v(k) (2.105) 
Substitution of Eq. (2.105) into (2.104) yields 
P(k-t,k-j|k.) = E{XCk.-'i|k-l)S ! (k-.j|k.-l) 

a ! ... 

- E {x (k-i | k-1) v 1 (k)} Kj (k) 

- K i (k)E(k)E£5t(k|k-l)X(k-j|k-l)} 

. -f K t (k)H(k)E£x0c|k-l)x' (k| k-l)}H' (k)Kj (k) 

■ • + K i (k)n(k)E{xCk|k-i)v , (k)}kj(k) 

- K^(k) E £v (k.) x 1 (L-j I k) } 

+ K. (k) E {v (k) x (k| k-1) } 11* (k) K. (k) 

^ J 

' • + K i (k)E{v(k)v* (k)}K (W). (2.106) 


X i • • • * J^l 
9 9 * 

(2.103) 










vanlfih „ ' ““ u c etrms 

Vanishes. Thus E,. , 2 

(2.106) may be wrj . tteji 

,N|k) “ p HHN 

* ' 


» 


» 

" ^ p (k, k-j | k -x) 

; x^ H Ck)P(k,k|k,-l)H,' ( k )K« (k) 

jW 

terinS and ^ C2.W7) becomes . . CanCelS “* and *Vt h 

PCk.-i.k.-j |y „ p(k ^' 

“ hlCfl C ° mpletes prooi ; of (2 . (2 . 
• it may be de °* *-3. 

ia a aero mean. G «*f fc&fl , k+I> k „ 

mean Ga Wss~M arkov , • ■ »k « o, 1, 

tot ° ** '0-MS) y lelds SeqUenCe ' Subst itutio n of Eq . (2 . 

xOc+l-i|k+l) n i( k+1 _ i | _ 

- k i(W)v(1 

t= o 1 ,k)x(k - I l k ) 

' rfJC+1,k)w(WJ ' « t to)vte) 

■f f. A 


( 2 , 


B y definition ,rm 1 » ••• , J 

AOlL w Ck) and v^k+n 

.*r—-■ --—»zz7r‘" c: -■- 
" *-*•*«**««>. 
independent of w(k) a „ d ov < J+ *>sequence whlch 

■ *«»> - •«. «„ 


•.; . ' - 

A r? J Y ' f * ‘ * " t 'V^T^ ii y^rri | • 




random n-vector independent of w(k) and v(k+l) for all k =• 0, 1,. 

In addition, [E{x(.-i| 0)), i -• 0, 1, ... , Jj is a zero mean random 

- *% * 

n-vector independent of w(k) and v(k+l) for all k «* 0, 1,/'... by defini¬ 
tion and Eqs. (2.19) and (2.20). However, x(i|o) is not independent of 

4 *' .• V,! 

{x(-i|0), i » 0, 1, ... , j) as is shown by (2.67). Therefore, it must 

be concluded that (x(k+l-i) k+1), i = 0, 1, ... , j)* is a Gauss-Markov 

* * * 

» 

-(J+1) sequence. This concludes the proof of Theorem 2.3. 

m 

2.6. 1 Estimation in Systems With. No Tima Delay 

It is of interest to examine the results of Theorem 2.3 for the 

* 

* 

case where there-is no time delay. This is easily done by setting J *= 0 

in Theorem 2.3 and results in the following theorem: 

* 

* *■ 

Theorem 2.4 , (a) The optimal filtered estimate x(k+l|k+l) 

for the-system described by Eq. (2.5) and Eq. (2.6) is 

t 

* 

given by the recursive relation 

x(i<:+l|k+l) « jl(k+l| k) + K(krf-l) [z(k+1) - x(k+11 k) ] (2.109) 

for k *= 0 , 1, ... where x(ol0) - 0. 

(b) K(k+1) is an n x m matrix which is specified 
by the set of relations ... , . 

K(k+1) - P(k+l|k)H.' (k+1 )IR(k+1)P(k+1 Jk)H 1 (k+1) + R(k+1)]"' 1 * 

(2.110) 

P (k+1 1 k) = $ (k+1, k) P (k | k) $ 1 (k+1,k) + r(k+l,k)Q(k)r’(k+l,k) 

(2.111) 

P(k+1|k+1) = [ I — K(k+1)H(k+1)]P(k+1Ik) (2,112) 

for k = 0, 1, ... , where I is the n x n identity matrix 
and P(0| 0) *• P(0) is the init ial condition for Eq. (2.111). 
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(c) The stochastic process'{x(kfl| k+1), k«0,l,,.,} 
which Is defined by the. filtering error relation 

5c(k-KL| k*KL) « x(kfl) - xOoKL|k+l) is a zero mean Gauss- 

/ 

Harkov sequence whose covariance matrix is given by 

* - 

Eq. (.2.112). 

Thus the results of Theorem 2.3 reduce to the familiar Kalman 

1 » . 

* , 

filter when no delay is present in the system. 

* ♦ „ 

2.7 . Computational Aspects 

One of the significant features of the filter developed in 
Sections 2.5 and 2.6 is its recursive form* The measurements can be 
processed as they occur and there is no need to store any measurement 

data. In fact, so far as storage of the measurement.and state is con- 

•* 

cerned, only {£(k-i|k), i = 0, 1, ... , J} need be stored in proceeding 
from time k to time k + 1. ‘ 

The information flow in the filter can be discussed by consider¬ 
ing the block diagram of Figure 2.2. 

Suppose that {x(k-i|k), i - 0, 1, ... , J) is known for some k 
and that it is desired to determine x(k+l|kKL) given z(k+l). The compu- 

r 

tational cycle would proceed as follows: 

1. The estimates {£(k-i|k), i = 0, 1, ... , J) are "propagated 
forward" by preraultiplying them by the state transition matrices, 
<J>^(k+!,k), The sum of these is added to (k+l,k)u‘(k), giving the pre¬ 
dicted estimate k(k+l|k). 
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< * 

* « 

* * 

■ 

2. £(k+l|k) is premultiplied by R(k+1) giving z (k+1 | k) which 

* * * * 

is subtracted from the actual measurement z(k+l) to obtain the measure- 

mmmammmmmmmmmm mm 

ment residual z(k+1Ik). 

——— —■ ’. t 

* 

3. The residual is premultiplied by the matrix Kg(k+1) and the 
result is added to x(k+l|k) to give x(k+l|k+l). At the same time, the 
delayed residual z(klk-l) is premultiplied by K^(k). The ith sum is 
added to various delayed sums as indicated in Fig. 2.2 to give 

‘ [x(k+l-i|k+l), i * 0, 1, ... » J}. 

4. fx(k+l-*i| k+1), I 0, 1, ... , J} is stored until the next 
measurement Is made at which time the cycle is. repeated. 

The interplay between prediction, filtering and smoothing is evi¬ 
dent at this point. It can be observed that each estimate is obtained 
using the other. The filter equations a,re 

J 

*(k+l|k)’ = z f,(k+l,k)x(k-i|k) + iKk+l,k)u(k) (2.113) 

i=0 x 

xCkxl-il k+1) * fc(k+l~i.|k) + ^(k+l) [z(k+1)-x(k+l| k) ] (2.114) 

To initiate filtering, {5c(-i| 0) - 0, i ** 0, 1, ... , J) is used 
• * 

and Eq. (2.114) can be solved. The equations then proceed recursively 
as described in the four steps above. 

Consider next the computation of the filter gain matrices 

{K.^(k+1), i - 0, 1, ... , J) and the three covariance matrices 

* 

P(k+1,k+l| k),{P(k-j,k+ljk), j = 0, 1, ... , J-l) and' {P(k-i,k-j| k), 
i,j =0, 1, ... , J). The relevant equations are • 


* 


f * 




. — . ... . — . -«. : & 3 


♦ 


* 
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J J 

P(k+1,fcML| k) «' a Z. $, (k+ljk)P (k-i,k-jI k,)4>! (k+1,k) 

t-0 j-0 * J 

+ r(kfi,k)q(k)r f (kri*i,k) (2.H5) 

' 

• # 

K^Ck+l) « P(k+l~i l k+l|k)H l Ck+l)[ECk+l)PCk+l,k+l|k)H , (k+l) 

+ RCk+1)]’' 1 , i * 0, 1, ... , J (2.116) 


J 

P(k->j,k+l|k) « Z P (k-j , k-i| k)4»! (k+1, k) , j * 0, 1, ... , J-l 

tR ° (2.117) 

P(k-i,k-j|k) « P(k-i,k-j | k-1) - ^(k) H (k) P (k,k-j | k-1) 


1, j « 0, 1, ... , J (2.118) 

for k = 0, 1, ... , with. PC-1,-jI 0) = E frC-i)** C-j)}; i,...,J. 

t * 

. A typical computation, cycle would proceed as follows: 

* * 

1. Given P(k|k), Q(k), Ct^(k+1,k) , i = 0, 1.J} and 

P(k+l,k); P (k+1, k+11 k) and {P (k-j , k+1| k), j = 0, 1, ... , J-l) are com¬ 
puted using Eq. C2.115) and Eq. ,(2*116) respectively. 

2. P (k+1, k+11 k) , fi? (k-j , k+1 | k) , k = 0, 1, ... , J-l), H(k+1) 

and R(k+1) are substituted into Eq. (2.117) to obtain (K, (k+1) , i=0,l,..,j) 
which, is used in Step 3 of the filter computations discussed in the 
previous paragraphs. 


3. P(k+l,k+l|k); (P(k-j ,k+l|k), j = 0, 1.J-l),' flC (k+1, 

i s 0, 1, ... , J) and H(k+1) are substituted into Eq. (2.118) to yield 
(P(k-i+1 ,k^j+l|k+1), i, j = 0, 1, ... t j). These values are stored 
until the time of the next measurement when the cycle is repeated. 

The matrix inverse which must be computed in Eq. (2.117) gener¬ 
ally poses no real problem. The matrix to be inverted is m x m, where 
m, is the number of elements in the measurement vector * In most systems 



'"•'V. 



B\ is kept small to avoid the high, cost oi; complex instrumentation.. 
Consequently, it is not unusual to encounter systems with 20 state 
variables but only 2 or 3 measurement variables. 


r t/‘ * -+*** 1 



CHAPTER III 

OPTIMAL CONTEIOL OF LINEAR DISCRETE-TIME STOCHASTIC,'SYSTEMS 

WITH. TIME DELAY 

♦ 

* * 

3.1 Introduction 

' i 

The. problem, considered in this chapter is that of controlling a 
system which. is subject to disturbances and measurement errors such that 
some, measure of the system’s behavior is optimized. It is shown, in 
addition, that the results may also be applied to the deterministic case. 
Neither, the stochastic nor deterministic results obtained here have ap¬ 
peared in previous literature. 

» 

Section 3.2 establishes the particular class of problems to be 
solved, the system model being that defined in Chapter 2. The perfor¬ 
mance measure is the expected value of a quadratic form in the state, 
and control variables over a fixed interval of time. The resulting 
problem is called the stochastic linear regulator problem . 

Section 3.3 reviews previous work in the area of control of 
time delay systems. The majority of the work has been done for open- 
loop deterministic continuous time systems. Very little appears in the 
literature on discrete-time systems and even less on the stochastic 
control problem for systems with time delay. 

Section 3.4 introduces the concept of optimality and applies it 
to obtain the solution to the stochastic control problem. In Section 
.3.5 the computational aspects of implementing the algorithm are presented 
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I.n Section 3,6 those results are. converted tp that for a deterministic 

* 

system and the separation principle of estimation and control is dis- 
cussed in light of these results. The separation principle states that 
the optimal control system consists of the optimal filter in cascade 
with the deterministic optimal controller. The result is also shown to 

reduce to the standard optimal controller when no time delays are pre- 

# 

sent In the system. 

3,2 Problem Formulation 

- -1--i - -i |_ _ - . I l». * 

. In this section the system model is presented as in .Chapter 2.* 
The quadratic performance measure of interest is defined. Physically 
realizable controls are discussed and the problem statement formulated. 

3.2.1 System Model 

The system model is defined by the relations 

J 

xChil) = E (p. (k+l,k)x(k-i) + rCk+1 ,k)w(k) • 4* ^(k+l,k)u(k) (3.1) 

i=0 

* . z(k+l) « H.(k+l)x(k+l) + v(k+l) (3.2) 

for k = 0, 1, 2, ... . The model is the same as that formulated In 
’Section 2.2.3 and for which the optimal estimation problem was solved. 
The following definitions and properties are repeated for reference, 
x ~ n vector (state) 

u =* r vector (control) 

w - p vector (plant disturbance) 

z - m vector (measurement) • 

v = m vector (measurement disturbance) 


f 
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.* 

6. « n k n (state transition matrix associated with the 

-L, , * 

delayed state vector x(k-i)) 

w. 

r «s n x p (disturbance transition matrix) 

... i 

r n x r (control transition matrix) 

» 

W x n (measurement matrix) 

’ $c(-i), 1=0,1,... ,J} ** zero mean gaussian random n vector with 

* , • / 

positive semi-definite covariance matrix 

Pt-i,-j) » EfcC-Dx'C-j)}, i,j=o,i.J 

(3.3) 

1*00 , k-0,1,2,... } « zero mean gaussian white sequence which. 

is independent of {x(-i), i=0,l,...,J} 
and has a p x p positive seraidefinite 
covariance matrix Q(k), kfO,l,.«. . 

(3.4) 

' Iv(k+1), k=0,1,...} » zero mean gaussian white sequence which 

is independent of* {x(-i) , i=0,l,..., J} 
and {w(k), k=0,l,...} and has an m x m 
positive semidefinite covariance matrix 

R(k+1) , k=0,l. (3.5) 

’{u(k), k=0,1,... } R control*sequence which is either knojm 

or can be specified as desired. (3.6) 
The following properties are recalled from Section 2.5.1 and are needed 
for the development of Section 3.4. 

1. fx(i), i = 0, 1, ...} is a Gauss-Markov-(J+l) sequence (3.7) 

2. x(i.) and.w(i) are statistically independent for all 
1 = 0, 1 , , . . . ■ 
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, . » 

*. 

« 

3. z(i) and w.(j) are statistically' independent lor all j>i, 

* X 

*“ 1,2 . ( 3 . 9 ). 

In addition to these properties, the system, equations (3.1) 

* 

and (3.2) must be "controllable," A discussion of "controllability" 
is presented in Appendix A. In general, a system is said to be control¬ 
lable, if, for the corresponding deterministic system, any initial 
state x(0) can be transferred to any final state x(t^) in a finite time, 

tf: 0, by some control u. If a system is not controllable, then there 

* 

is no guarantee that a control sequence (u(k), k = 0, 1, ...} can be 
found which transfers the system to some desired final state from arbi¬ 
trary* initial conditions. 

* 

3,2.2 Performance Measure 

A control sequence {u(k), k - 0, 1, ...} is to be constructed 

/ 

to control, the state (x(k), k - 0, 1, ...} of the system over some 

* 

fixed interval of time [0,N], N - positive integer, such that the per¬ 
formance measure 

K. ' * . . 

N 

J N - E{ I [x’(i)A(i)x(i) + u f Ci-l)BCi-l)u(i-l)]} (3.10) 

i=l 

is minimized. A(i) and B(i) are, symmetric positive semidefinite matrices' 
which are n x n and r x r respectively and E{*} denotes the expected 
value operation. The expectation is over x and u. 

Jjj is usually interpreted as a "system error plus control effort" 
measure of performance. The first term on the right-hand side of Eq. 
(3.10) implies that the desired state is zer*. If at each point i, the 
desired state is some arbitrary x^i). then x(i) would be replaced in 
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r * 

Eq. (3.10) by x(i) - x^i). 

« 

Although, x (i) - 0 for the system model of this paper, this is 

* 

not a.requirement for obtaining a solution. Unfortunately, if x^(i) ^ 0, 
the mathematical development is quite complicated and obscures the basic 
results. This is obvious from the results obtained by Williams^ for 

the case where no time delays are present. Thus, as a matter of mathe¬ 
matical convenience x^Ci) is chosen equal to zero. Note that the quad¬ 
ratic nature of the term implies that the pleasure of error here is one 

» 

• of error-squared and actually of weighted-error-squared because of the 

freedom in choosing A(i)« 

♦ * 

. The second term of Eq. (3.10) is sometimes called "control energy" 

■***.. . 

as a consequence of the quadratic nature of the term. As with the first ‘ 
term it is referred to as "weighted control effort" because of the arbi¬ 
trary nature of B(i-1). 

Thus J N may be viewed as a measure which provides for a trade¬ 
off between'system error and control input. The relative importance of 
the two terms is reflected in the choice of A(i) and B(i-l). Because 



trol sequence {u(i-l), i - 1, ... , N) which minimizes J^. 
312.3 Physically Realizable Controls 


The control sequence {u(i-l), i - 1, ... , N} which minimizes Eq. 
(3.10) is not arbitrary. For example, the solution may lead to control 

sequences which cannot be mechanized in practice^such as those which 

* 

require input data that is not physically available when required. Ad¬ 
ditionally, since it is anticipated that variations in the system's 




state occur, it is. desirable to have the control sequence depend upon 

information which. is available about the state, namely, the measurements. 
* » 

* 

•Thus the control sequence Is to Involve feedback. If no plant distur¬ 
bances were present and if the initial conditions were perfectly known 
then an open-loop control law would be satisfactory, assuming, however, 
the plant is precisely known. Plant and measurement disturbances, how¬ 
ever, in. the system described by Eqs. (3.1) and (3.2) make a feedback con 

* * 

trol law necessary! In the sequel, the.control sequences depend only 
upon information about the system's state which, is available for pro¬ 
cessing. 

• For any given k « 0, 1, ... , N-l it is obvious that the avail¬ 
able data on the system's state consists of the sequence of measurements 
£z(l), ... , z(.k)} and the mean value of the delayed initial states 
{x(-i), i = 0, ... , J). The control vector at k can then be written in 
the form * 

u(k) = p k Iz*(k), x*(0)J (3.11) 

where z*(k) is the mk vector 
z*(k) = ztOl 


z(k)J 

and x*(0) is the n(J+l) vector 


(3.12) 


x*(0) = x(0) 
x(-l) 





;3#8P 



ancj, y, is an r-dimensional vector-valued function of the indicated vari- 

k 

ables. Note that y, is to be determined such, that J vt is minimized and 

. k N 

it is-not necessarily restricted-to be of the same form for all k. 

- / 

A control vector of the type defined by Eq. (3.11) which depends 

only on available data is a physically realizable control and y^» • 

* 

k ** 0, 1, ... , N-l is a physically realizable control law . For k *= 0, 
uCO) can only be a function of x*(0) since no measurements are available. 
If independent of z*(,k) for all k, then yCk) is an open-loop con¬ 
trol law;. , , 

* *►#»». .♦ 

3.2.4 Problem Statement 

The problem, can now be stated as follows. 

"Determine a physically realizable control law of the form 
(3.11) for the system described by Eqs. (3,1) and (3,2) 

which, minimizes the, quadratic performance measure (3.10)." 

i t 

Such a control lav; is called an optimal control and the problem 
itself is called the discrete stochastic linear regulator problem (with 
delay). The word "regulator H arises because x^Ci) - constant. 

V 

3.2.5 Discussion , 

Three important restrictions on the class of problems are: 

1. The performance index is time independent. The terminal 
time itself may not be part of the performance index. 

2, No amplitude bounds are placed on the control vector. How¬ 
ever, the second’ term in tends to’limit excessive control. 



. c 
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3. The state at the terminal time is not constrained. As with 

* 

the control vector the state at the terminal time may only be affected 

I * 

.indirectly through judicious selection of A(i) in the performance meas¬ 


ure. 


3.3> Review of Previous Work on Optimal Control of Linear Systems with 

Time Delay . *• % , 

* 

» 

Attention is restricted below to work appearing in the litera¬ 
ture which may be applied to the stochastic control problem with time 
delays in the plant. 

Thus, the large number of papers which develop open-loop control 

* 

are omitted. Similarly, those papers which employ unity feedback and 
vary only the gain in the forward path are not discussed. Briefly, then, 
the papers discussed below have the following characteristics: 

1. Use of state-variable feedback 

2. Presence of delays in plant 

* 

The reader interested in other characteristics of the control of time 
delay systems is referred to the extensive bibliography compiled in 
the bibliography. 

A number of approximate techniques appear in the literature 

(Bibliography: 73, 116, 118, 157, 159, 161, 164, 212). These methods 

7 "“ST 

use either a Pads approximation to the time delay which expresses e 

as a ratio of polynomials in s or they express the delayed state x(t-h) 

as a Taylor Series. 

xCt-h) = x(t) - hx(t) + h 2 j!(t) - ... 

2 ! 


i 
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Once either of these approximations is made, the problem may then be 
treated as one with no delays and the well-knoim optimal control results 
may be applied. Since, by definition, these methods are approximate, 
they yield a sub-optimal solution and are not discussed further. It is 
recognized, however, that such methods may be very good, depending on 
the accuracy of the approximation. 

♦ , 

In 1969, Eller et al developed an exact deterministic control 

law for continuous systems with time delay. The solution, however, is 

*• 

similar in form to the estimation solution by Kwakernaak^ mentioned ear¬ 
lier and suffers from the same computational disadvantages. * Both results 
are limited to plants with a single delay, although the authors suggest . 

i 

that the theoretical results may be easily extended to include multiple 
delays. It should be remarked, however, that even for a single delay 
the computation is so excessive that with a large computer (CDC 6600) 

i 

only scalar examples are worked. 

» • * 

g 

In 1969 Koivo derived the solution to the stochastic control 
problem for continuous systems with time delay. He showed it to be the 

same as the result obtained by Eller et al, thus verifying that the 

» 

•separation principle also holds for continuous systems with time delay. 

■ ' » * ♦ • * 

Larson and Wells q overcame"some of the computational problems 

in a paper published in 1969. Attention is restricted to serial systems, 

t 

where the delay is in the forward path only, but this represents a large 
number of practical problems. Their results obtained are optimal only 
for single input-single output systems, however. 
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? 

* » 

» 

The expanded state representation for discrete systems with 
time delay presented in Chapter 4 of this dissertation was first intro¬ 
duced by Koepcke 10 in 1964. An alternate form of the expanded state 
representation was later used by Day 11 in 1968. The results achieved 
by both authors require extensive computation and storage. 

The results obtained in the remainder of this chapter can also 

* 

be obtained by examining the submatrices of the solutions of Koepcke and 

* * 

Day. Solutions of these submatrices requires considerably less computa¬ 
tion and storage time, 

3.4 Stochastic Control Problem 

* " 

3.4.1 Problem Formulation 

* 

From Eqs, (3.1), (3.10) and (3.11) the problem becomes 
J 

x(k+l) ~ 2 <j>. (k+l*k)x(k~i) + r(.k+i,k)w(k) ^(k+l,k)u(k) (3.l4) 

z (k+1) = H(k*KL)x(k+l) + v(k-KL) 

N 

J N = E{ 2 x'(i)A(i)x(i) + u 1 (i-l)B(i-l)u(i-l) (3.15) 

i«l 

u(k) = n k U*(k),x*(0)] (3.16) 

i 

where z*(k) is the ink. vector 

(3.17) 
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and x*(0) is the (J+l) vector 

4 

X*(0) = x(0) 

> • 

* 

x(-J) 



t • 

(3.18) 


In the deterministic case* the expected value would be removed 

from the expression for J^, and the measurement process would become 

, * 

z(k+l) = H(k+l)x(k+l). Assuming the system is observable.the state var- 

7 . 

iables could be calculated exactly, and the uncertainty associated with 

i 

the delayed initial states would be removed. The estimates 
{x(-i), i=0, 1, ... , J) would be replaced by the actual values of the 
delayed initial states, 6c(-i), i=0, 1, ... , J}. The stochastic regu¬ 
lator problem may now be stated. 

, j| .... ‘ ■ 

'j\ "" 

3.4.i Problem Statement 

Determine a control law of the form (3.16) for the system of 
Eq. (3*14) which, minimizes the performance measure in Eq. (3.15). 

The resulting system has the block diagram which is given in 

* 

Pig. 3.1. The problem is to specify the controller which will operate 
upon the output states, z (k+1), to determine the control vector which 
minimizes the performance measure, in general, the resulting control 
law could involve feedback of all the preceding values of the measure¬ 
ment vector. Prom a computational point of view this poses a similar 
problem to that of estimation discussed in Section 2.5. By applying the 
Principle of Optimality to'this problem (discussed in the next paragraph) 
a set of recurrence equations are found which resolve this problem quite 
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handily. Another problem is that of determining the optimal controller 

when not all of the states are available for measurement. The separation 

1 2 

principle, which was first suggested by Kalman and Koepcke- and later 

13 14 

proved by Joseph and Gunchel provides a neat solution to this qtiss~ 
tion. This important result reduces the optimization problem to two 
separate optimization problems, one of estimation, the other of control. 
It states that the optimal controller is the same as the deterministic 
controller which operates on the optimal state estimates as' if they were 

i 

the actual values of the states. In this chapter the separation princi¬ 
ple is modified slightly and also shown to apply to the delay case. The 

* * 

stochastic control solution is shorn to rely upon the availability of 
the optimal estimates of the delayed states for implementation. Subse¬ 
quent comments are made to show that the deterministic controller and 
the stochastic controller are the same. 

3.4.3 Principle of Optimality 

The principle of optimality may be stated as follows: 

■ •» f y\ 

Theorem 3.1 . For any initial state and initial con¬ 
trol, the- remaining optimal control at any subsequent 
time must constitute an optimal ,one for the remainder 
of the trajectory. 

A simple interpretation of the principle of optimality would be 
as follows. Suppose that, for some discrete-time system whose initial 
state is x(-i), the optimal control {u_(t), t j t j t } minimizes some 

V v I Z 

. . $) ■ 

performance measure J over the interval [t^, t£3• Then the principle of 






optimality states that the control {u°(t), t® jZ t s t^} minimizes the 

same J for the same system over the interval [t 1 , t 2 ] with the initial 

* 

.state x(t®)*which resulted from u°(t) acting over the interval [t,, t']. 

/ J" 

*• 

A proof of Theorem 3.1 is given in Meditch** (p. 331) and is a 

A 

powerful result for use in the solution of control systems optimization 
problems. In discrete-time problems with no delay, the problem can be 
reduced from one of determining an entire control sequence at once to 
one of determining the control as a function of a state of time k based 
on the results at time k + 1. For the discrete time problem with delay 
the same technique again results in a set of recursive equations for the 
control. 


3.5 Stochastic Control Problem For Systems ‘With. Time Delay 


is defined tp.be the minimum value of the performance measure 
J N in (3.15), 



= min . e. 
u(0) 


N 

min E { £ x'(i)A(i)x(i) + u®(i-l)B(i-l)u(i-l)} 
u(N-l) 1=1* 

(3.19) 


The-problem is one of selecting rN variables, namely u(0),u(l), ... , 

u(N-l) to minimize J . A Lagrangian formulation would require the solu 

N 

tion of rN algebraic equation subject to the constraints expressed by 
the system equation (3.14). Even for modest problems this approach de¬ 
mands excessive computation. 

The problem can also be viewed as an N-stage decision process 

where the N decisions, u(0),u(l), ... , u(N-l) minimize the quadratic 
• * » 
cost. By applying the principle of optimality the decisions are made 
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one* at a time, rather than simultaneously, the Nystage problem is re- 
duced to N one-stage problems. This technique starts with the final 
stage of control and uses induction to proceed backwards in time to an 
arbitrary initial time as shown below. 

3.5.1 Single-stage 

Suppose that the problem is simply that of selecting a control 

* 

which minimizes the performance measure for the last stage of control. 

That is, the problem is a single-stage optimization problem 

* » 

V* min Efe'(n)A(N,x(N) + u’(N-l)B(K-l)u(N-l)} (3.20) 

1 uCN-1) 

However, from Eq. (3.14) 

J 

x(N) = £ <f>.(N,N-l)x(N-i) + F(N,N-l) w(N-l) + ij;(N,N-l)u(N-l) 

1=0 1 (3.21) 

Substituting this result into Eq. (3.20) and dropping most time indices 
for convenience 

J J 

V. = min E {[ £ x 1 (N-i-l)({>! + u’ip’ + w*r']A[ £ <J> x(N-j-l) 

1 u(N-l) i=0 1 j=0 3 

+ ^u + Tw] + u f Bu] (3.22) 

By noting that the individual product terms are scalars and that A is 

symmetric, the terms may be combined to yield 

J J J 

V = min E‘ {£ £ x* (N-i-l)$ !A|> .x(N-j-l) + 2uM>'A £ <$> .x(N-i-l) 

1 u(N-l) i=0 j=0 13 i=0 1 

J 

* + 2w’r'A £ <j> .x (N-i-1.) + 2u*i|> f Arw + u'^'M+Bju 

i-0 

+ wT 1 ATw} 


(3.23) 



I 

I 

r 
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Where the Indicated expected value is over x, w and u. Making use of 
the matrix identity 

✓ 

trace (ABC) = trace (BCA) = trace (CAB) 
and since each term, in Eq. (3.23) is a scalar^the third and fourth terms 
of Eq. (3.23) vanish 

J J 

E{w*r*A Z <t>.x(N-i-l)} = E{tr[r'A Z <p .x(N-i-l)w’ (N-l) ]} 

i^O i**0 x 

* * 

J \ 

« trace[F'A Z <{> E (x(N-i'-l)w 1 (N-l)}) 
i=0 1 

** 0 by property (3.8). (3.24) 

Similarly, Etu^'ATw} = u'^'AFECw} » 0 (3.25) 

by properties (3.6) and (3.4). 

From Chapter 2 the property of conditional expectation gives 
E{x} = E{E(x|y)}, where the outer expectation on the right-hand side is 
over y, thus allowing Eq. (3.23) to be rewritten 

J J 

V *= min E{E[ Z E x'(N-H)^'AM(N-j-l) 

1 u(N-l) i=0 j=0 1 J • 

J 

+ 2u’^'A Z (j).x(N-i-l) + u 1 [i|j’A ip + B]u 
i=0 X 

+ w f r*Afw z*(N-l),x*(0)]} (3.26) 

This equation can be minimized by minimizing the inner expected value 
in Eq. (3.26) with respect to u(N-l) for all z*(N-l) and x*(0). The 
physical realizability condition requires that u(N-l) be some determin¬ 
istic function of z*(N-l) and x*(0). Thus the second and third terras of 
Eq. (3.26) become respectively 
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J 

■E{2uVA E <f> x(N-i-l) z*(N~l),x*(0)} % 

1=0 1 • 

’ J . 

= 2u f ^'A E <p E{x(N-i-l) z*(N~J),x*(0)} (3.27) 
1=0 1 

and E {u* [i|j'Ai|j + B]u z*(N-l),x*(0)} = u* [i/j *Ai^/ + Bju (3.28) 

Now, setting the gradient of the inner expected value of Eq. (3.26) equal 
to zero, 

J 

•3 v x = 0 = 2fVA E <f. i E{x(N“i“l) z*(N-l),x*(0)} 

Su(N-l) i=0 • 

+ 2ty f AiJ> + B]u(N-l) ' (3.29) 

and solving for u(N-l), expression (3.30) is obtained. 


u(N-l) = -[i|)’Atj> + B]“Va E <j> -x(N-i-1 JN-l) (3.30) 

1=0 


where ip - ^(N,N-1) 

<f> i - <|>^(N,N-1) 

B = B(N-l) 

A = A(N) (3.31) 

A 

Notice that the separation principle is evident in this one- 
•stage case. The optimal control is a set of gain matrices each of 
which is associated with a. separate filtered estimate, where the gain 
matrices and filters are computed independently. Define 

W Q0 (N) = A(N) and (3.32) 

note that Wqq(N) = Wqq(N) since A(N) is symmetric. 

S i (N-l) 3 -{-/A^ + B]^'A^, 


(3.33) 
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J 

and write u(N-l) = Z S.(N-l)*(N-i-l|N-l) (3.34) 

1=0 1 4 * 

may now be evaluated by substituting Eq. (3.31) into Eq. (3.23) (less 

the third and fourth terms). 

* 

J J 

V = E{ Z Z x* (N-i-l)<|> *W nn <|> .x(N-j--l) 

1 1=0 j =0 1 

J J . 

- 2 E Z &’(N-i-l|N-l)<|»!W 00 ^[^*Wip+BrVw n * x(N-j-l) 

1=0 j=0 00 3 


1=0 j=0 1 00 

• [^ , W 00 nB] _1 f'W 00 4> j *(N-j-l|N-l) + w'r'W 00 rw} (3.35) 

'* ■ • 

This expression may be simplified by noting that 

* 

*(N-i-l|N-l) = x(N-i-l) - x(N-i-l|N-l) (3.36) 


Then, letting A . denote the n x n matrix 

i3 


. 4j s < t’l w do ,f ' [ ^ ,w oo’f’ +B]_:L ^' w oo < ' , j 


(3.37) 


The second and third terms may be combined in the following way. 

J J 

Z Z [-2x , (N-i-l|N-l)A.x(N-j-*l) + *’ (N-i-l|N-l) A .&(N-j-l | N-l) ] 
1=0 j=0 


J J 

Z Z [-2(x!-x!) A.x. + (xj-xj) A. .(x.-x^)] 
i=:0 j =0 i 1 ‘ « 3 i 1 iJ 3 J 


J J 

2 2 [~2x! A. .x. + 2x’A x. + x! A. .x. 

i=0 j=Q x iJ 3 1 iJ j 1 J 




- x! -A st + x! a .*.)] 
i i3 3 1 13 3 


(3.38), 
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The second, fourth and fifth terms cancel since each term is a scalar, 

'* * 

*= A* , and the equation results in a double sum where each sum con- 
ij 

tributes the same range of indices. Thus Eq. (3.38) reduces to 

1 £ (x! A - xJ A x.) 

i =0 1=0 1 j j 


(3.39) 


Therefore Eq. (3.35) may be written 

J J 




J J 


+ E{ 1 lx’ A ) + ^{w’r’W^rw} (3.40) 

£_0 j —0 ^ 

Since the last two terms of Eq. (3.40) are scalars, these terms become 


J J 


E{ 1 l x! A £ + w'r’w nn rw} 

i=0 j=0 1 « j 00 


J J 

E{trace[ 1 1 x! A, .X. + w’r'W^rw]} 

i=0 j=0 ** J 


00 


J J 

E (trace [ 1 £ A—X.x' + r'W nn rww f 3} 

i=0 j=0 3 31 uu 

J J 

trace[ 1 I, A P + TW rq) 

i=0 j=Q ^ J uu 


(3.42) 


where P . is the n x n filtering covariance matrix 

ji 

P - S P (N-j-l ,N-i~l | N-l) =E{x(N-j-i)x , (N-i-l)> 

•J" 

and Q = Q(N-1) £ E {w(N-l)w' (N-l) } 

Denote Eq. (3.42) as a(N-l) and define 11 . (N-l) as 

' MijCN-l) - c n tW 00- W 00«^' W 00 4 ' +Bl '^ ,W 00 ] ^ 


(3.43) 

(3.44) 

(3.45) 



Replacing the time arguments, the above results may now be summarized 

» 

for the single-stage problem. 


u(N-l) « Z S (N~l)£(N-i-l|N~l) 


(3.46) 


i«0 


S t (N-l) = -ir<K,N-X)W 00 (N)i(>(N ( N-l) 

+ (N,N-l)H 00 (Kf)«f i (N,N-l) 


(N) » /A(H) 


1 « 0, 1, .a. | J 

i *= j = 0 . 

elsewhere 


J J 

« E{ £ Z x' (N“i-l)H i | (N-l)x(N-‘j-l)) + a(N-l) 
1 i-0 j=0 J 


(3.47) 

(3.48) 

(3.49) 


(N-l) - *!<N,N-lj {W 00 (N)-W 00 (K)^(l\;,N-l)[^ , (N,N-l)W 00 (N)^(N i fl 4) 

+ B(N-l) ]“V (H,H)W 00 (N)}^ (N,N-1) 


j 


(3.53) 


- ♦*(N,N-I)W 00 (N)« (N,N-1) + ^<N,N-1)W 00 (N) 

• ^(N,N-1)S (N-l) 

J 

J J 

a(N-l) = trace { Z Z, A.(N-l)P(N-j-l,N-i-l|N-l) 

1=0 j =0 i - ) 

+ r'(N,N-X)U 00 (N)r(N,N-l)Q(N-l)} (3.51) 

. (N-l) = Sj(N-l)[ii-'(N,N-l)W 00 (N)iKN,N-l) + B(N-1)] -1 S (N-l) 

(3.52) 

where the initial conditions are 


W i:j (N+l) = 0, a(N) = 0 
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i 

e 

* *■ 


3.5:2 Double-Stage Problem 

' i i.. t ‘ ‘ inrrni,r ■« 

* 

The two-stage problem may now be written 

V 9 « min min E {[x*(N~l)A(N-l)x(N-l) - u'(N-2)B(N-2)u(N-2)] 
u(N-2) u(N-l) 

+ [x 1 (N)A(N)x(N) + u* (N-l)B(N-l)u(N-l]} (3.53) 

where the expected value is over x(N), x(N“i), u(N-l), u(N-2). Note 

that u(N-l) and u(N-2) are required to be physically realizable. Util- 

* 

izing the principle of optimality, Eq. (3.53) may be rewritten 

min E{x , (N-l)A(N-l)x(N-l) + u'(N-2)B(N-2)u(N-2) + V.) 

1 u(N~2) . 1 

(3.54) 

From Eq. (3.49) and (3.51) it can be seen that 

■ 

J J 

E {V-} « E{E[ 2 2 x f (N-i-l)M..(N-l)x(N-j-l) + ct(N~l]} (3.55) 

i-0 j~0 ^ 

J J 

« E{ 2 2 x*(N-i-l)M .<N-l)x(N~j-l)} * a(N-l) (3.56) 

i=0 j-0 13 

» 

since the inner expectation of Eq. (3.55) is eVer x* 

Thus Eq. (3.54) can be written 

V 9 = min E{x’(N-l)A(N-l)x(N-l) + u*(N-2)B(N-2)u(N-2) 

1 u(N-2) 


J J 

+ 2 2 x*(N-i-l)M..(N-l)x(N-j-l)} 4- a(N-l) (3.57) 

i=0 j=0 13 

J J 

- min E{ 2 2 x'(N-i-l)W..(N-l)x(N-j-l) 

u(N-2) i=0 j-0 XJ 

» 

+ u’(N-2)B(N-2)u(N-2)} + a(N-l) (3.58) 

where ct(N-l) is taken out of the minimization procedure since its value 
does not depend on u(N-2) and the following definition is used 
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W (IKL) .5 


(N-l) + A(N-l) 


1 - j * 0 


M..(N-X) 
*1 


85 o,i, . • •» j 

(except i « j « 0) 


0 

V. 


i.j >0 


(3.59) 


Since W and M are symmetric W.. - W* and M.. = M;This definition is 

ij ji rj ji 


a matter of convenience for later development. Because of the system 
equation (3*14) 


x(k+l) « E $ (k+l,k)x(k) + ip (k+l,k)u(k) 4- r(k+l,k)w(k) (3.60) 
i=0 1 


u(k) can only affect x(k+l). Therefore, since 


tion of Eq. (3.58) may be rewritten 


W* , the minimiza- 

J 


V 0 = min E {x'(N-l)W nn (N-l)x(N-l) 
2 v(i<-2) UU 


J-l 


+ 2 E x f (N-l)W . (N-l)x(N-j-2) 

j«0 °>3 +1 


+ u , (N-2)B(N-2)u(N-2)} (3.61) 
Substituting the plant equation (3.60) into Eq. (3.61) and momentarily 
dropping the time indices 


J J 


V - min E{ E E x!*!W 0*t x + +. 2uE <J>.x 
2 u(N-2) i=0 j=0 1 1 U J 3 OO i=0 


+ 2w'r’W Art E <j>. x. + 2u’^’W rw + u* *W iM-B]u 
00_._ n 1 ' J- ■ 0Q 00 


i=0 


00 1 


J-l J 


J-l 


+ w ,r 'w 00 rw + 2^ .r^-ii 0 . iJ+1?i + 2 ^f o u, ^ ,w o,j-iXj 


J-l 

+ 2- E w * r 1 . , x. } 
j=0 


0,j-l j 


(3.62) 
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As in the development of Eqs. (3.24) and (3.25) for the third, fourth 
and ninth, terms vanish. Because the terms x^ « {x(N-i-2), i = 0, ... , J) 
occur at or before N-2, they do not depend on u(N-2). Thus the minimiza¬ 
tion of (3.62) depends only on the second, fifth and eighth terms. 

Utilizing the properties of conditional expectation as in equa¬ 
tions (3.23) through (3.26), Eq. (3.62) becomes 

J J J 

V 9 “ min E{E[ £ £ x J-$.! w nn < h x i + 2u*'p* w 00 2 $i x i 

1 u(N-2) i=0 j“0 1 1 00 3 3 i-0 ■ 


+ u , [ty*'W 00 if>+B]“ 1 u + w'r’w^rw 

j-i j j-i 

+ 2 £ £ x!«J>]W n . ..x. + 2 £ u^’W x z*(N-2) ,x*(0) ] } 

js=0 1=0 x - J j=0 •? 

(3.63) 


Now Eq. (3.63) is minimized if the inner conditional expectation 


E {■ 


z*(N-2),x*(0)} is minimized with respect to u(N-2). Since 


x^~ x(N-i-2) then x^ is unaffected by the minimization, .Thus the first 
and fifth terms of Eq. (3.63) are unimportant. Similarly the fourth 
term is unaffected by the choice, of u(n-2). By setting the partial der¬ 
ivative of the inner expectation (less the first, fourth and fifth terms) 
with respect to u(N-2) to zero 

J J-l 


2 


_^V,_ = 0 = E{’/W on £ (jX, + t4-'W <|H-B]u + *' £ W x 

u(N- 2) i-0 1 00 i=0 °' 1+1 1 


Z° ,X rt / 

(3.64) 


By solving for u, this becomes 

i-l 


J-l 


u(N-2) = -L^ , W 00 ^+B]’-> , [W 00 £ $. ± * + £ W Q i+1 * ± ] 

i=0 i=0 * 


(3.65) 



Define S^-2) = -[W^ + W 01+:L ] i = 0, ... , J (3.66) 

and recall from Eq.. C3.59) that s 0 if 1 or j > *J 

uCN~2) « E S.(N-2)x(N-i-2|N-2) 
i-0 1 

may now be evaluated by substituting Eq. (3.60) and Eq. (3.65) into 

Eq. (3,57), omitting the third, fourth and ninth terms which have been 

: * 

shown to vanish. 


J-l J-l 

V_ *=■ min E{ E E x!W.,, x. + E E x!<|>!W__<f> .x. 
2 u(N-2) 1=0 j=0 1 i+1 >j+l j 1=0 j=0 11 00 J J 


J J 


J J 


"4*Mw [ *V B] ' v ^oV 

3 


J-l J 


- 2 


J 1 0 *i W i , i+1* t* ’ 'W +B ] '’ "oo* J X J 


J J 


- 1 , 1 , 


+ e e; fti*Jw’ M’Woo^+B] r ! w 00 <M, 

j[=0 j-0 J J 


J J-l 

+ 2 E E x!<f)!W .. x. 
1=0 3=0 11 °*J +1 J 


J J-l 
3 


“l,;.«i 


+ 2 E E X£*IW 00 *U' , W ftn lP+B] >'W ft. 
i= 0 -i=0 1 1 00 00 °»3+l 3 


J-l J-l 


11 


+ ii 0 + ,B o, J+ i*i 

~4o ^ s WW [ *' B 00* +B1 “ 1 *’ W 0,J+1*J 


J-l J-l 


“l,v 


- 2 E E ftX ur^Wy+B] f% X. 

1=0 j=0 0 » 1+1 00 0,j+l j 



a - ■> 


+ w f r!w 00 rw 

+ a(N-l) 
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(3.67) 


#■ 

This definition of W expressed by Eq. (3.59) allows all upper limits on 
the sums to be J. Combining terms, Eq. (3.67) may be rewritten 

' V ’ “ Jo /=o E{x i‘Vi, j+ i + UVj + ♦i w o.j + i>V 


*” .j. ii 


+ E{&^[-2(f[W 00 i|<[« , W 00 <M-B] «f- 1 W 00 4> 


~ 2W 0 , i+l’'’ f ^ ’ W 00 ' f,+B ] _1 ’P' 


j 


00'j 


- 2*XJ’« , w 00 'H-b] , 


I 1 


i 00 


0,j*KL 


" W 0.H4* I * ,H 00*- 1B )“ 1 *. ,W 0, j +l I V 

+ %,i + i^^V +B] ^ ,K 0, j+ i 
+ Z^W^+BrVy.^}*.} 


+ Efa'r’W^rw) + a (N-l) 


(3.68) 


Substituting = x^ - X in Eq. (3.68) and noting that 

Etx’yx) = tr E{yxx’} - 0 since the term in parentheses is a scalar 






J J 


r 2 " jf 0 A Eix i [W l+l,j+l + '’’i'Wj , + V'O.j+l 


- 2 ^V [ ^cA Br V'Wj 

- 2Wi ii+1 *(rWoo*+ B rVWoof, 

- 2*iV 00 *[#-W 00 t»]“S'W 0ij+1 

+ *iW 0 0^*' W 0 0 ^B ] "^ ,W 00^ 

+ W 0,1- 1 « 1 ^'V +B] '^ ,w 0, j+ 1 

+ 2*i» 0 0*I* , «pO* + B]-S'W 0ii+1 J^} 

+ K q ^+B ] -1 ^'W 0[) <f. 


+ W < >1+1 *l*' W OO^B) >'W 0>j+ l 




+ 2 *i w oo^ ,w oo* +B] ♦’"o.j+dV 


+ E{w’r'W 00 rw} + a(N-l) 


Define, 4j = [^W Q0 + W^Hf'W^l[W 0>j+1 + W^.] 


Substitution from Eq. (3.65) yields 


. \i ■-■ S'^'W^+Bl-is 


The second expectation of Eq. (3.69) may be written as 
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J J 

E{.E X x|A..x' } 
t-0 J-0 1 * j 


(3.71) 


Define My + *iVj + *l W O lj+ l 

“♦iV t *' w oo+ +Brl * ,w oo*j 


- V^’Vi+i 


(3.72) 


Substitution of Eq. (3.66) into Eq. (3.72) yields 
M ij " W i+l,j+l + ^i W 0,j+l 

+ I *l w oo + w t+i,o 5 ^j + * S j< k)1 
. J J 

By letting a(N-2) = E{ X X xJA.x. + wT'W Fw + a(N-l) 

i=0 j=0 x 3 U0 


(3.73) 


J J • 

- $ X tr(A,P..) + tr(r , W n FQ) + a(N-l) 
. i=0 j-0 iJ 00 

‘ » 

The expression for V may now be written from Eq. (3.69) as 

* 

J J 

• V 0 =*E:U X. x!M, .x.) + ot(N-2) 

2 1=0 j =0 1 i; > 3 


(3.74) 


(3.75) 


The results for the two-stage optimization process may now be summarized 
with the time indices restored in Table 3.1. 
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Table 3.1 


Summary of Optimal Control Algorithm for 2-Stage Sequence 


u(N-2) - Z S (N-2)£(N-i-2|N-2) 


(3.76) 


i-0 


S (N-2) « ~[i|j 1 (N-l,N~2)W nn (N-l)ip (N-l,N-2) + B(N-2) J"* 1 


^(N-i) , 


J J 


00 

• i|» , (N-l,N-2)[W 00 (N-l)<j> i (N-l > N-2) + W 0#i+1 ( N “D 1 

(3.77) 

i - j = 0 

i,j - 0, ... , J except i~j-0 
0 ' i,j>J ’ (3.78) 


A(N-l) + M (N-l) 
ij 

M ±j (N-l) 


V = Ei Z Z x 1 (N-i-2)M..(N-2)x(N-j-2)} + a(N-2) 
2 ±=0 j=0 3 


(3.79) 


(N-2) « [(}>^(N-1,N-2)W 00 (N-1) + W ±+1 (N-l)3 [<\>. (N-l,N-2) 

+ t|/ (N-l, N-2) S. (N-2) J * 4 (N-l, N-2) W n ,,, (N-l) 


+ W -,, n -4.1 < N " 1 > 

i+1,3+1 
J J 


(3.80) 


a(N-2) = trace[ Z Z A. (N-2)P(N-j-2,N-i-2IN-2)] 

i-0 j=0 lj 

+ trace[ V 1 (N-l,N-2) W Q0 (N-l) V (N-l,N-2)Q(N-2)] 


+ a(N-l) 

^(N-2) = S!^(N-2) U >' (N-l >N-2 )W oq (N- 1)0 (N-l ,N-2) 


(3.81) 


+ B(N-2) ]’" 1 S^ (N-2) 


(3.82) 
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Despite the complex nature of the various equations^ note that 

the computation for the two-stage process is quite straight forward. 

First, the feedback gain matrices {S (N-2), S (N-l), i-0,,*... , J} are 
* i i 

determined and they are used in conjunction with the optimal filtered 

estimates {£(N-i-2|N-2),£(N-i-l|N-l), i=0, ... , J) respectively, to im 
* 

plement the control signals. Thus the separation principle is again 


apparent. Comparison of Eqs.(3.76) through (3.82) with the results for 

> 

the single-stage case, Eqs. (3.46) through (3.52), the results are seen 
to be identical except for the change in time index. In effect, Eqs. 
(3.76) through (3.82) reduce directly to Eqs. (3.46) through (3.52) by 
recognizing that Wqq(N) is the only non-zero term of W for the single- 
stage case. 


3.5.3 t-1 Stages 

Assume now that the results of the two-stage case may be gener¬ 
alized to t-1 stages. The equations characterizing the control as ex¬ 
pressed in Eq. (3.76) through (3.82) then become as in Table 3.2." 

3.5.4 t Stages < 

From the principle of optimality 

V = min E (x' (N-1+1) A (N-t+1) x (N-1+1) 
t u(N-t) 

+ u'(N-t)B(N-t)u(N-t) + V t _ 1 > (3.90) 

Examine from Eq. (3.86) and express it in a slightly different 

form so that the minimization becomes more evident. 
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Tabic 3.2 

w 

Summary of Optimal Control Algorithm for a (t-1)-Stage Sequence 


u(N-t-l) « E S.(N-t-l)*(N-t-i+l|N-t+1) 
1=0 x 


(3.83) 


S^ (N-t+1) «= -tyHN-t+2,N-t+l)W 00 (N-t+2)i|j(N-t+2,N-t+l) + BCN-t+l))" 1 

•i//(N-t+2,N-t+l)[W 00 (N-t+2)<{. i (N-t+2,N-t+l) 4* W 0>i+1 (N-t+2)] 

1 * 0, 1, ... , J (3.84) 


W ij ( N "t+2) .*= (N-t+2) + A(N-t+2) 


1=0, 1| ... , J 
1 = 5=0 


Hy (N-t+2) 


1,5 = 0, ... , J except 1=5=0 




(3.85) 


M (N-t+1) = (N-t+2,N-t+l)W 0() (N-t+2) + W i+1>0 .(N-t+2)3 

• [<f> (N-t+2,N-t+1) + i p (N-t+2,N-t+1) S . (N-t+1) ] 
j 3 

+ <^( N-t+2,N-t+l)W 0>j+x (N-t+2) + W i ,, 1>j+1 (N-t+2) 

1,5 = 0, 1. , J (3.86) 

J J 

V -..1 = E IE £ x'(N-t-i+l)M J .(N-t+l)x(N-t-j+l) + a (N-t+1) . (3.87) 

*' * i=o j=0 *3 , 

f 

J J 

t* (N-t+1) = trace[ 21, E Aj . (N-t+1) P(N-t-j+l,N-t-i+l| N-t+1)] + a (N-t+2) 

1=0 j=0 

+ tracelr’(N-t+2 > N-t+l)W 00 (N-t+2)T(N-t+2,N-t+1)Q(N-t+1)] 

0 . . ... 

♦ . . . 

(3.88) 

; A i . (N-t+1) = S jL (N-t+l) 1 (N-t+1,N-t+2 )VJ (N-t+2 )ip (N-t+1,N-t+2) 

^*3 ...» 

+ B(N-t+1)] _1 S^(N-t+1) 

l,j =0, 1, ... , J (3.89) 

-v/here M. = M!. and A - = A!, and W.. - W!. 

ij J-*- ’ J 3 - ’ jl ij j 1 
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J-l 


■V. « min E £x 1 (N-t+l)M x(N-t+l) + 2 S x' (N-t+l)M_ , x(N-t-j) 

tKL u(N-t) 00 j«0 °»J +1 


J-l J-l 

+ E E x f (N-t-l)M.,, x(N-t-j)} + a(N-t+l) (3.91) 

t «0 j =0 i+l, 3+1 


Since the expectation is over x, w and u, E {V } *= V. , as expressed in 
* . t*"l a 

Eq. (3.90). Substitution of Eq. (3.91) into Eq. (3.90) yields 

V t « min £{x’(N-t+l)[A + M 00 ]x(N-td-l) + u f (N-t)Bu(N-t) 
u(N-t) 

* 

J-l 

+ 2 E x*(N“t+.1,)M .-x(N-t-j) 




'O.j +1 


J-l J-l 

+ E E x'CN-t-DM^, .^xOi-t-j)} + a(N-t+l) (3.92) 
i«0 j«0 


i+l,j+l 


Once again, using the properties of conditional expectation, Eq. (3.92) 
becomes 


V « min E {E x'(N-t+1) lA+M_]x(N~t+l) + u* rN-t)Bu(N-t) 
* u(N-t) 00 


+ 2 ^ o x ' (N ' t+i;,M 0 l j+l X j + t f Q j f 0 X i M l+l,j+l X j 


z*(N-t), 


x*(0) } + a(N-t+l) 

• * 
vhere x^ « x(N-t-i). Expansion of the inner expectation and substitution 

of Eq. (3.14) and (3.85) in Eq. (3.93) yields 

<• 

M,, + A i - j = 0 

i,j = 0, ... , J (except i = j = 0) 
elsewhere ( 3 . 94 ) 


W, . 


iJ 


*r 


J J 


V * njji E{ E Z x i^i W 00^1 x 1 
* u(N->t) 1*0 H U0U J J 


2u, ^ W 00 X *i x i 
UU i«0 1 


+ 2w f r l W 00 £ <|>+ 2uVU 00 rw + uM^’Wqq^+BJu 
1*0 

J-l J J-l 

+ W'r'W„n™ + 2 X X x.'if'W. ,,x + 2 X u'i|i'W n ,.,x 
00 t=o t=o 11 0>j+1 J J=0 °*J +1 i 


+ 2 X w'r'Wn ....x, z*(N-t),x*(0)} + a(N-t+l) (3.95) 
j=0 0,;1+1 J 

# 

♦ 

As' shown by Eqs. (3.24) and (3.25), the third, fourth, and ninth terms 

v 

Vanish, and a(N-t-f-l) does not depend on u(N-fc) * The time indices are 


x^ « x(N-t-i) 
u = u(N-t) 
w * w(N-t) 

B •* B(N-t-l) 


* 1 * ♦ t (N-t+l,N-t) 
i It « )|»(N~t+l ,N~t ) 

W lj ~ W ij < N ~ t+1) 


Observe that- the first and sixth, terms are unaffected by the choice of 
u(N-t) and minimize Eq, (3.95) with respect to u(N~t). 


■ V 3 V « 0 * [2^’W n0 Z cf> x + 2<|/ 1 Z W x z*,x* 
- . 3^1) \ °°1=0 1 1 1=0 0>;i+1 j 

+ 2l^'W 00 ^+B]u 

By. solving,for u and restoring the time indices 


<0)1 


(3.96) 


u(N-t) = X S.(N-t)£(N-t-i|N-t) (3.97) 

1=0 1 

Sj^N-t) = -t<c'(S-t+l,M-t)W (S-t+DtKN-t+l.N-t) + B(N-t)] -1 


• ip' (N-t+X) [K (N-t+lH (N-t+l,N-t) 

+ w o,i+i (N - t+1)3 


(3.98) 





* - 
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Now V must be evaluated from Eq. (.3.91) 
t 


V.. « E{x<(N-t+l)W x(N r t+l) + u'(N-t)B(N~t)u(N-t) 
t 00 


J-l 

+ 2 E x* (N*-t+l)VL .,_x(N-t-j) 
j«0 u »J +i 


* 


J-l J-l 

+ E E x*(N-t“t)Wj 
i«0 j*=0 


x(N-t-j) + a(N-t+l)} (3.99) 


i+l,j+l 

Substitution of the system equation (3.14) into Eq. (3.99) as well as the 

* * 

expression (3.97) for u(N-t) yields 

* 

j j j j 

v t - E{ £ £ x!*!W * x + 2 £ xlS'i^'W £ * x 

* 1=0 j=0 1 1 00 J J 1=0 1 1 °?j=0 j j 


J J 


,-l. 


+ E E ^S'[fW n J+B] S x 
1=0 j=0 1 00 11 


+ u'r'\i QO rvj 

j-i j * 


j-i j 


+ 21 E xI^'VT x + 2 E E £*S’tJrW * x. 
■ j =0 i=0 ** ** ^ * j—0 i=0 ^ ^ 0,j+l 1 

J-l J-l 

+ E E xl-W. - • . ,,x,} + a(N~t+l) ’ 

j-f\ *_A 1 3L+l,j+l J 
i-u j—u 


(3.100) 


Once again, since W - 0 for i or j > J, the upper limit on the sums 

i»j 

may be J for each sum without changing the expression (3.100). Therefore, . 

Eq. (3.100) may be rewritten, letting x^ - ~ and noting that 

Efx.x!) - 0 for all 1, j. 

X 3 


it 
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*.“o *JVj + 2S ^Vi+ s i»’v* B3! 


j j 

* 

< 

r 

+ ♦±*o l j+i + s ?' w 0 , j+ i + w i +1 , j+1 x j 

j j 


'j 


+ 2 : S x« S![fw x, 

i-0 j=0 x 00 33 

+ Elw'r'vr^rw 


x 4 ) 


00 

+ a(N-t+l) 

*. * 

where the time indices are 


x. *= x (N-t-i) 

• i 

(j» t « ^(N-t+l.N-t) 

S 1 « S(N-t) 

„ * 

iff « ^(N-t+i,N-t) 

W.. = w..(N-t+1) 
ij 3-J 

r « r(N-t+i,N-t) 


w = w(N-t) 


Define , . = S^^’W^+B]" 1 ? 

J J 


- a(N-t) = trace l Z.A.. P.. + r'W n rQ] 

i=0 j=0 13 13 UU 

+ a(N-t+l) 

then Eq. (3.100) becomes 

J J 

V « E{ X. Z x!M. .x.} + a(N-t) 
i=0 j-0 1 3 - 3 


(3.101) 


(3.102) 


M tf S ^00 + W i+l,0 ][ *j + '+■ ^i W 0, j+ l + VlJ41 < 3 ’ 103 > 

J J 


(3.104) 


(3.105) 


But this is the same result as Eq. (3.86) with the time index changed. 

It remains to be demonstrated.that, in fact, the u(N-t) expressed 
by Eqs. (3.94), (3.96) and (3.102) minimizes the performance measure 





given by Eq. (3.90), Recall from, differential calculus that the van¬ 
ishing of the gradient with respect to u, which led in the general case 
to Eq. (3,96) is only a necessary condition for V to be a minimum. 

That is, u(N-t) in Eq. (3.97) only guarantees that V. attains a station¬ 
ary value. A sufficient condition that V attain a minimum is that the 
second gradient of V. with respect to u(N-t) be positive. This condi- 
tion is determined by examining the gradient of Eq. (3,96). 

J J . 

u&o “ m ' w oo^ A + *’£* 4 +* + [ ^V- ]u i < 3 - 96) 

The "second gradient" of Eq. (3.96) becomes 

V u<N-t) [V u(N-t)< V t>l =*' W 00* +B 

Thus a sufficient condition that a minimum be obtained is that 
the matrix 

i|>* (N-t-l,N-t)W 00 (N-t)i]>(N-t-l,N-t) 4* B(N-t) 
be positive definite for all t = 1, 2, ... , N. 

The control algorithm and associated performance measure equa¬ 
tion’s may now be written, letting k = N - t. These results are summar¬ 
ized in Table 3.3. Block diagrams of the optimal controller and the 
optimal control system are.shown in Figures 3,2 and 3.3 respectively. 

The control law expressed by Eq. (3.105) requires the optimal 
estimate of the delayed states at each stage. The form of the control 
law indicates the following theorem: 
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Figure 3.3 Block Diagram of Overall Optimal Control System 
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Table 3.3 

*• 

Summary of Optimal Control Algorithm for k~Stage Sequence 


u(k) = Z S,(k)S(k-i|k) 
i«=0 

S t (k) -[t> ’ (k+1, k)W 00 (k+lH (k+1, k) + BCk)] -1 

•*'(k+l,k)[tt 00 (k+l)<f, i (k+l,k) + W 0>i+1 Ck+l)] 


(3.106) 


V ti (k+1) - M (k+1) + A (k+1) 

i M ij (k+D 


i ** 01 1} . ■ . ) J 
i=j=0 

i,j*Q, ... , J except i=j=0 


(3.107) 


(3.108) 


M^Ck) « [^|(k+l,k)W 00 (k+l) + \ +lt0 ^+l))[^.iM i k) + ^(k+l.^SjCk)] 

+ 4»[(k+l.k)W 0>j+1 (k+l) + U. +li . +1 (k+l) 


i»j e i> ... t J 


J J 


V ’ i, = E { £ E X 1 (k-i)M . (k)x(k-i)} + a GO 
N i =0 j =0 x 3 . 

J J 

a(k) « trace{ E E, A.(k)P(k-j,k-i|k)} + a(k+1) 

1=0 j =0 

+ trace {r , (k+l,k)W 00 (k+l)r(k+l,k)Q(k)} 
■ 4. (k) - S[(k) [4,' (k+1 ,k)H 00 (k+l)^ (k+1 ,k) + B(k)]" 1 S j (k) 
• ijj ~ 0) lj ... ) J 


(3.109) 


(3.110) 


(3.111) 


(3.112) 


for k - N-l, N-2, ... , 0,.where a(n) = 0, WqqCN) = A(N), and (N) is 
zero elsewhere. The r x r .matrix [ip f (k+l,k)WQQ(k+l)\f;(k+l J k) + B(k)] is 
required to be positive definite for all k. The minimum value of the 
performance index, for (N-k) stages of control is given by Eq. (3.110;. 


*■ & 
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Theorem 3.2 The optimal control system for the sto- 

* 

chastic linear regulator problem consists of the 

optimal linear filter cascaded with the optimal 
» * 

feedback gain matrix. The parameters for the two 
parts of the control system are determined inde- 

* * 

pendently. The performance measure is governed by 

0 

Equations (3.109) through (3.112) where the boundary 
condition is a(N) =0. 

The next section verifies that the gain matrix is the same as 
that for the deterministic controller. This confirms the separation 
principle as applied to time delay systems. 

3.6 Discussion of Results 

■ » 

3.6.1 Comparable Results for Deterministic Case 

• It is interesting to relate the results expressed in Eqs. (3.106) 
through (3.112) to the deterministic case where no plant disturbances or 
measurement errors are present. For such, a case the noise covariance 
matrices, R and Q, are identically zero. The estimates of the states 

V.." 

are equal to the states themselves forcing the filtering error covariance 
matrix to zero and the expected value operation is over a deterministic 
quantity, yielding the quantity itself. The resulting equations (3.106) 
through (3.112) are presented in Table 3.4. 

Note that the computation of the control law expressed by Eqs. 
(3.113) through (3.119) once again verifies the separation principle. 
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Table 3.4 

* • 

Summary of Optimal Control Algorithm 
for Deterministic Time Delay Systems 

* J 

J 

u(k) « E S.Ck)x(k-i) * (3.113) 

i*=0 

S t (k) - ' (k+1 ,k)W Q0 (k+1)^(k+1 ,k) + BCk))- 1 

* , Ck+l # k)[W QO (k+l)^ t Ocrl-l,k) + W Q t+1 (kfl)] 

® * 

i* 0, 1, ... , J (3.114) 


Hy (k+1) 


r 

= M (k+1) + A (k+1) 
j My (k+1) 

0 


i^j-O 

*■ 

i,j“0, ... , J except i=j=0 
elsewhere (3.115) 


(k) - [^(k+l,k)W 00 (k+l) + w i+1 (k+DJf^Ck+l.k) + iKk+l,k)S (k)] 

+ (^(k+l,k)W 0ij+1 (k+l) + w. +ljj+1 (k+i) 

1,J “ 0, 1.J • (3.1X6) 

J J 

V„ . = X X x' (k-i)M.. (k)x(k-l) (3.117) 

1=0 j=0 1J 

a(k)* - 0 

,,A. (k) = don't care 

i/j 


(3.118) 

(3.119) 
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• * 

That Is, the control law is identical to that expressed by Eqs, (3.106) 
through (3.108) except that x(k-i|k) is replaced by x(k-i). Thus the 

controller in the stochastic case treats the optimal estimate of the 

* 

states as if they were the actual values of the states. 

The value of the performance index is not the same, however. It 
is, as expected, less for the deterministic case than the stochastic case. 


3.6.2 Comparable Results for Stochastic No-Delay Case* . 

* * 

If’no delays are present in the*system, the results expressed by 
Eqs. (3.106) through (3.112) should reduce to the standard optimal con¬ 
trol law. The transformation may be made by letting J c 0 and writing 

the resulting equations which are presented in Table 3.5. These results 

* * 

are, in fact, identical to the standard optimal control results such as 
those obtained by Meditch^ Chapter 9. 


3.7 Computational Aspects of Optimal Controller 

The recursive nature of the computations required to generate 

the optimal control sequence and to evaluate the performance index is 

evident from Eqs. (3.106) through (3.112). 

1. Given (W..(k+1), i,j=0,l,...,J) compute (S.(k), i=0,l,...,J) 
■*•3 i 

from Eq. (3.107). 


2. Substitute the values of (S^(k), i=0,l,...,J} and WQQ(k+l) 
into Eq. (3.112) to obtain ■{A. . (k), i,j=0,l,...,J}. 

■^■3, •* 

3. Substitute {A (k), i,j=0,1,...,J}, W^^k+l), a(k+1), and 

the error covariance matrix {P(k-j,k-i|k), i,j=0,l,...,J) into Eq. (3.111) 

to obtain a(k). 

• .... 



f 
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Table 3.5 

■4 

* 

Summary o£ Optimal Control Algorithm ior Stochastic Systems 


With No Time Delay 

* 

* * 

* / 

u(k) = S 0 (k)S(k|k) ' ' (3.120) 

S 0 (k) - -[*' (k+1.k)W 0Q (k+1)tji(k+1.k) + B(k)] -1 

» 

•(■•(k+l.WW (k+l)$ 0 (k+l,k) , . (3.121) 

* • 

W 0Q (k+l) = M (k+1) + A(k+1) ' (3.122) 

Mo 0 (k) = 4> 0 (k+l,k)W 00 (k+l)[(. 0 (k+l,k) + Kk+l,k)S 0 (k)] (3.123) 

V K _ k - E(x'(k)M 00 (k)x(k)) + a(k) (3.124) 

* 

a(k) ® a (k+1) + trace {I* * (k+l,k)W 0 Q(k+l)rCk+i,k)Q(.k)} 

. * 

+ trace(A 0 g(k)P(k| k)) . (3.125) 

,A 00 (k) •= S’(k)[J<’(k+l,k)W 00 (k+l)^(k+l,k) + B(k)] _1 S 0 (k) 

» -(.^(k+l,k)U 00 (k+1) (k+1, k) S Q (k) 


(3.124) 
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4. {S^(k) * 1*0,1,...,J) and {W^ (k+1) , i,j**0,l,.., ,J) are sub~ 
stituted into Eq, (3.110) to obtain {M^(k), i,j«0,1,...,J). 

5. Substitute, (k), i,j»0,i,...,J) into Eq. (3 V 110) to obtain 

W 

6. Substitute {M. (k) 4i i, j«*0,l,..., J) and A(k) into Eq. (3.108) 

* ■ J' 

to determine {W^ (k) , i, j*=0,l,... ,J). 

•* 

The cycle is then repeated, letting k become k~l in the above 
procedure. Note that although {M^(k),,i,j-0,1,...,J) must be computed 
at each stage, is not necessarily of interest and, therefore, need 

only be computed at k = 0 to obtain the minimum value of the performance 

* 

measure for all N stages of control. The values of a(k) and 

.(A^j(k) , i,j^0,l,.,J) must be computed at each stage, however, because 

. a 

of their inter-relationship and the dependence of on a(k). Note 

also that the error covariance matrix from the optimal filter must be 

♦ 

known at each stage. If only the optimal control is required then steps 
2, 3 and 5 may be omitted. 

The optimal control is physically realizable since it is simply 
a linear transformation on the estimates of the delayed states. The con¬ 
troller in Fig. 3.2 is a set of time varying gain matrices 

{S, (k), 1=0,1,...,j}. 

1 * 

Since the computations proceed backward in time, it is clear 
that the time history of (S^(k), i=0,l,...,j} must be determined prior 
to system operation. That is, it must be precomputed and stored for 
later use. 



CHAPTER IV 


v EXPANDED STATE REPRESENTATION AND DUALITY' 

a * 

4.1 Introduction 

a 

In this chapter an alternate representation of discrete-time 

* * 

systems with time delay is formulated. The system equations (2.6) 'and 
(2.16) are imbedded in an "expanded state" form. This representation, 
discussed in detail in Section 4.2, expands the state dimension of the 
system In direct proportion to the magnitude of the time delay. Once 
the time delay system is expressed in the expanded state form the solu- 

, J c 

*tions are well-known * since this form does not express explicitly 

the time delay dependence. Unfortunately, since the resulting system 
dimension is directly proportional to the time delay magnitude, the re¬ 
sulting computation required for a solution is often considered too ex- 

. * * 

tensive for practical application; As a consequence the expanded state 

i t*r~* 

form does not receive much attention in the literature. 

The major results of this chapter are obtained using the expanded 

* • ♦ 

'State formulation. One consequence of casting the time delay problem in 
the expanded state form is that the resulting matrices have a large num¬ 
ber of null elements. Thus, rather than work with expanded matrices (as 
previous authors have done), it may be more efficient to partition the 
matrices. The solutions to these submatrix equations are obtained in 
Sections 4.3 and 4.4. The results are identical to those of Chapters 2 
and 3. In Chapter 5 the computational savings are demonstrated, by 


comparing the partitioned results to those obtained by working with the 

#. 

entire matrices. 

In Section 4.5 the expanded state form is used to,demonstrate 
the duality of estimation and control for time delay systems. Duality 

i 

follows as a natural consequence of being able to express a time delay 

. . • 

system as an expanded system with no time delays. It is also shown, how 

* 

ever, that duality can not be extended to the partitioned solutions to 
the estimation and control problems. 

* 

4.2 Expanded State Representation of Systems with Time Delay 
The system equations of Chapters 2 and 3 are 

J 

x(k+l) - E $. (k*M,k)x(k~i) + ip (k+l,k)u(k) 4* r(k+l,k)w(k) (4.1) 
i=0 

z(k4-l) * H(k+l)x(k+l) 4- v(k4-l) (4.2) 

■9 ' * 

These equations may be also written as an expanded state representation 

x(k+l) « <|> (k+1 ,k)x(k) + x(k+l,k)u(k) + r(k+l,k)w(k) (4.3) 

* .... « 

z(k+l) = J((k+l)x(k4-l) + v (k+1) • (4.4) 

where the following definitions apply 

$(k4*l) *= x(k+l) 
x(k) 







* t{> ** n(J+l) x r (J*KL) control transition, matrix 

, * 

H «* mCJ+1) x n(J-KL) measurement matrix 

» ■ 

* # $* 

xCO) ^ zero mean gaussian random n(J*fl) vector with positive 
semidefinite covariance matrix E(0) 

» 

* * 1 * . 

fw(k), k=0, 1,. ♦.} « zero mean gaussian white sequence which is 

independent of x(0) and has a positive semidefinite p x p 
covariance matrix Q(k), k-0, 1,... 

Jv(k*H) ,k«0,l,.. .} « zero mean gaussian white sequence which is 

independent of x(0) and (w(k) , k-0,1,...} and has a posi- 

* , 

tive semidefinite m x m covariance matrix R(k-KL), k+0,1,.. 

. , t 

k«0, 1,... } = control sequence which is either known or can 
be specified as desired 

4.3 Optimal Estimation Solution of Expanded State Representation 

With the system described by Eqs. (4.3) and (4.4) having the 
properties expressed in the preceding section, the solution is well- 
known (Meditch ** Chapter 4) for this is the standard estimation 
problem. Once again, of course, the system must be ’’observable" as 
described in Appendix A or the estimation problem cannot be solved. 

Under the assumption of observability, then, the following theorems 
from Meditch , (pp. 174-177 and p. 356) are stated for a system 
described by Eqs. (4.3) and (4.4). 
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Theorem 4.1 If the optimal filtered estimate x(k|k) 
and the covariance matrix E(k| k) A of the corresponding 
filtering error &(k|k) = 3z(k) - &Ck|k) are known for 
come k » 0, 1, ... , then 

a. The single-stage optimal predicted esti¬ 
mate for all admissible loss functions is given by 
the expression 


fc(k+l|k) = $(k+l,k)$(k|k) + $(k+l,k)v(k) 


(4.13) 


b. The stochastic process {ft(k+l|k), 
k ~ 0 l4 l, ...} defined by the single-stage prediction 
error* &(k+ljk) = &(k+l) - x(k+l k) is a zero mean 
Gauss-Markov sequence whose covariance matrix is 
given, by the relation 


$*(k+l J k) = $ (k+1, k)'£ (k | k)$ 1 (k+1, k) 

+ ^(k+l,k)^Ck)? , .(k+l,k) 


( 4 . 14 ) 


Theorem 4.2 • 

a. The optimal filtered estimate x(k+1|k+1) 
is given by the recursive relation 


j(k+1|k+1) = &(k+l|k) - fc(k+l) [z(k+l) - fKk+l)$(k+l|k)] 

. , 15) 

for k = 0, 1, ... ,•where x(0[0) = 0 


b. • k(k+l) is an n(J+l) xm matrix which is 
specified by the set of relations 

k(k+l) = ^(k+l[k)^' (k+1) [ft<k.+l)^Ck+l|k>ft* (k+1) + l((k+l)3‘' 1 

(4.16) 

I*(k+1 [k) = <|>(k+l,k)^(k[k)<J> ’ (k+l,k) + ?(k+l,k)$(k)?'(k+l,k) 

(4.17) 

£(k+l|k+l) = [I-fc(k+l)fr(k+l)]$(k+l|k) (4.18) 

for k = 0, 1, ... where I is the n(j+l) x n(J+l) identity 
matrix and l*(0|0) = 2?(0) is the initial condition for 
Eq. (4.17). 

c. The stochastic process * (a: (k+1 k+1), k=0, 1.... } 
which is defined by the filtering error relation 

k (k+1|k+1) = x(k+1) - $(k+1|k+1) (4.19) 

k=0, 1, ... , is a zero mean Gauss-Markov Sequence 
whose covariance matrix is given by Eq. (4.18). 
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For 'these theorems, the following definitions hold 

* 

» ■* * t 

$ Ck+11 k) s E ft Ck+11 k)%' Ck+lk) } 

where P(k+lj'k) is n(J+l) x n(J+l) 

S(k+l|k) = x(k+l|k) 

*(k|k) 


(4.20) 


(4.21) 


*(k-J+l|kjj 

*' r 

where £(k+l|k) is n(J+l) x 1 . 

The use of Eq. (4.20) allows £(k+l|k) to be expressed in terms of its 
submatrices as 


J* (k+11 k) « 


wm 

P(k+l,k+l|k) P(k+l,k|k) . . . P(k+l,k-J+l| k) 

P(k,k+l|k) P(k,k|k) 

• * • * • 

• • • 

[p(k-J+l,k+l|k) P(k-J+1,k|k),,.P(k-J+l,k-J+l|k) 


(4.22) 


Similarly, 

1*0*1 k) = 


P(k,k|k) P(k,k-l|k) . P(k,k-j|k) 

P(k-1,k|k) P(k-l,k-l|k) 

• • 

• • 

P(k-J ,k| k) .P(k-J,k-rj| k) 


(4.23) 


4.3.1 Partitioned Representation of the Expanded State Solution to 
the Estimation Problem 


The solution to the optimal estimation problem for systems with 
time delay is expressed by Eqs. (4.13) through (4.18). As mentioned 
earlier, however, the dimensions of the matrices involved in this expanded 
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state solution may render the computation prohibitive. The appearance 
of a large number of null elements in the defining equations for these 
matrices [Eqs. (4.5) - (4.12), (.4.22), (4.23)] indicate that partitioned 
solutions may be advantageous. Thir= is, in fact, the case as Eqs'. (4.13) 
through (4.18) are examined below in partitioned form. 


Eq. (4.13) may be expressed in partitioned form as 


x(k+l|k) 

xCklk) 


4*o $1 • • • 

I 0 . .0 

x(k|k) 

*<k-l|k) 

i 

0 

u(k) 

• 

• 

• 

it (k+1-J J k) 


0 I 

• 

* 

• 

• • • • 

0 . . 0 *1 0_ 

• 

• 

• 

• 

it (k-J | k) 

9 

9 

9 ■ 

* 0 

0 

(4.24) 


The expansion of Eq. (4.24) in partitioned form yields 

J 

x(k+l|k) = z 4 (k+l,k)X(k~i|k) + 4 (k+l,k)u(k) (4.25) 

i«0 1 • 

t 

which is identical to Eq. (2.66) obtained in Chapter 2. 


Eq. (4.15) may be expressed in partitioned form as 


it (k+11 k+1) 

S(k+l|k) 

V 

M-.HO ... 0 4 it (k+1 Ik) 

it(k|k+l) 

• • 

= *(k|k) 

• 

- K i 

• 

X(k|k) 

• 

• • 

• • 

it (k+l-J | k+1) 

• 

• 

it (k+l-J | k) 

• 

• 

K J- 

• 

• 

it (k+l-J |k) 


« *(k+l|k). - K o (k+l)[z(k+l) - H(k+l)5t(k+l|k)J 

• * 

9 

■ 9 ' 

. • 

it (k+l-J |k) - K.j(k+1) [z(k+i) - H(k+l)5t(k+l-|k) ] 

(4.26) 
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The partitioned result of Eq. (4.26) may be expressed as 

*(k+l~i|k+l) « St (k+l-i.|k) - ^(k+l) [z(k*fl) - U(k+l)R(k+l|k)] 

0 

(4.27) 

which is identical to Eq. (3.90) obtained in Chapter 2. 

Eq. (4.16) may be expressed in partitioned form 

Kg(k+1) 

Kjft+1) 


K (k+1) 

J 



P 00‘ * * P 0J 

“ 


J O ... 0, 

p oo * * 

• p o/ 

H'" 

V 

+ R 







sz 

• • 

• • 

0 

• 



• 

• 

• 

• 

* 

0 

• 


* 

• • 

pi p 

L\jo ,p jjJ 

• 

• 

• 

0 


1 

• • 

pi 

L p jo * • 

• 

pi 

• p jjj 

• 

0 

t, 


r oo H ' 

P 10 H ' 


P J0 H ’j 


[HPJ 0 H' + R] -1 


(4.28) 


where P 


ij 

H 


E {x(k+l-ijk)x'(k+l-j|k)} = P fc+l-i,k+l-j|k> 
H(k+1) • 

R(k+1) 


Restoring the time indices to Eq. (4.28) the partitioned gains become 
KjXk+1) - P(k+l-i,k+l|k)H f (k+1) [H(k+l)P(k+l,k+l Jk)!! 1 (k+1) 

+ R(k+1)]“ 1 (4.29) 

which is identical to Eq. (2.100) obtained in Chapter 2. 

To ease notation below, the following time indices are used 


P? . = P(k+l-i,k+l-j|k) 

■*- » J 

P i,j “ P(k-i,k-j|k) 
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Examination of th& partitioned form, of Eq. *29) yields 




Restoring the time indices, the following submatrix relationships result 
from Eq. (4.30) 

J J 

P(k+1,k+1|k) = E E «{) i (k+l,k)P(k,k|k)<|>!(k+l,k) 

1=0 j = 0 

+ r(k+l,k)Q(k)r'(k+l,k) (4.31) 

J 

P(k-j,k+l|k) = E P (k-j , k-i I k) ! (k+1, k) j=0,l,..., J-l .(4.32) 

i=0 

P (k+1,k-j|k) = P'(k-j,k+1 1 k) (4.33) 




Jf. r 


•» ~ •* ii 
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Expressions C/*.31) and (4.32) correspond to (2.103) and (2,110) respec¬ 
tively'. 

* 

v Equation (4.18) is now examined in terms of.its submatrices, 

« t 

* 

dropping the time indices 


pi pi 

*0,0 • * * *0 # J 


10 ... o’ 

• 


K o" 

HO ... 0 

p3 p3 

*00 * ' ' *JJ 

• • 

• • 


ox. 

• • 

- 

K i 

a 


. • 

a a 

• • 

’ p i,o * 


a * 

« • 

0 ... 01 

m 


*i. 

• 

a a 

. P J,0 • • • P J,J. 


0 


• I 

X - K q Hi 0 . 

- X.H! 10 .. 0 

X i 

-K H '01 

2 • 

I • • 

i 

• * « 

• la 

i 

• | 


- Kj H j 00 


A\- V ]p ooj . [1 " V ]p oi: 


0 

0 I 


r p 3 

00 


10 


01 


• a a 


. P 


0J 


P 3 
L JO 


• • • f t 


JJ J 




10 


^00 ! P 11 


L jo 


• I 

. » 

I 

K j llp ooj P J1 


K,HP^ » 
1 01 • 


« . . • 




_i___ 

K j HP oii • : 


!p 3 

• 1J 

. J.- 

!,- 

» jj 


- K„HP 


1 0J 


K J HP 0J 


(4.34) 


Restoring the time indices to Eq. (4.26) the submatrices of P(kjk) may 
be expressed as 

P(k-i+1,k-j+11k+1) = P(k+l-i,k+l-j|k) - K i (k+l)H(k+l)P(k+l,k+l~j|k) 


(4.35) 











which. Is Identical to the result {Eq. (2#111),] obtained earlier using 
properties of conditional expectation. 

In summary, then, the results of Chapter 2 expressed by Eqs. 
(2.86), (2.61), (2.108), (2.109), (2.110), and (2.111) have been obtained 

in this section as shown by expressions (4*25), (4.27), (4.29), (4.31), 

* »• 

(4.32) and (4.35) respectively. 


4.4 Submatrix Representation of Expanded State Solution to Optimal 
Control Problem 

, The optimal control solution is well-known (Meditch 5 , Chap¬ 

ter 9) for the system described by Eqs. (4.3) - (4.12), for this is the 
formulation of the standard discrete-time optimal control problem. Once 
again, of course, the system must be ‘'controllable” as described in 
Appendix A or the control problem cannot be solved. The familiar optimal 
control solution is stated below in Theorem 4.3* This is followed by 
examining the submatrices of the solution equations. These submatrix 
results are identical to the optimal control solution obtained in Chap¬ 
ter 3. ’ 

If the system is controllable the following results from Meditch, 
p. 356, can be stated. Recall that the stochastic linear regulator 
problem is that of minimizing the Performance Index expressed as 


P.I. = E [x*(i)X(i)x(i) + u 1 (i-l)B(i-l)u(i-l)] 
i=l 

where x and u are defined by Eqs. (4.5), (4.6) and 


(4.36) 


A(k) = 


A(k) 0 ... 0 

0 

» • 


n(J+l) x n(J+l) 


(4.37) 



fe(k) « B(k) 

r x r 

* 

Theorem 4.3 , For a system described by Eqs. (4.3) 
and (4.4) the optimal control system for the stochas¬ 
tic linear regulator consists of the optimal linear 
filter cascaded with, the optimal feedback, gain matrix 
of the deterministic linear regulator. The parameters 
for the two parts of the control system are determined 
separately. The performance measure for the complete 
control system is governed by 

* 

9 

V N-k K jS£x* (k)M(k)x(k)} + a(k) and 

< * * * » 

a(k) *= a(k+l) + tr[r r (k+l,k)ft(k41)?(k+l*k)?{(k)] 

f * * * * 

- tr$(k+l,k)ft(k+l)$(k+1 ,k)£(k)P<k | k) ] 
where the boundary condition is a‘(N) *= 0 

Theorem 4.4 The optimal control law for the stochastic 

linear regulator problem' is the linear feedback control law 

* » < 

u(k) - 5>(k)x(kjk) * (4.41) 

where the r x n(J+l) feedback control matrix £>(k) is 
determined recursively from the set of relations 

W(k+1) = M(k+1) + A(k+1) ^ (4.42) 

• • * . * * 

2f(k) = (k+l,k)W(k+l)^(k‘!-l»k) + B(kfl)]~* 

4* (k+l,k)ft(k+l)^(k+l,k) (4.43). 

M(k) * (k+l,k)ft(k+l)|(k+l > k) + (k+l,k)W(k+l)^(k+l,k)S(k) 

(4.44) 

S ' • . . M 

The procedure is now to express Eqs. (4.39) through (4.44) in 
terms of their submatirices to obtain the results of Chapter 3. First 
the following definitions are made to enable a relationship to be estab¬ 
lished between the results pf Chapter 3 and those obtained below. In 

: . - v -. 

the performance measure, Eq. (4.39), .(k) is associated with the state 

■ U. : . - 

vector x(k-i) and x(k-j) of the form 


95 

,(4.38) 

(4.39) 

* 

(4.40) 
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x * (kri)M (k)x (k-j) 

so that the. following definition holds 
M(k) | M 0() (k) . . . . M 0j (k) 


(4.45) 


l M j0 (k) • • i • Mjj Ck)j 


(4.46) 

Similarly, in Eq, (4,40), w(kd-l) may be expressed in partitioned form as 


St(k+i) = 


W 00 (k+1) . . , .W oj (k+l) 


LWjO^+1) 


W (k+1) 
JJ - 


(4.47) 


where W(k+1) is n(J+l) x n(J+l) and W..(k+1) is n x n. 

‘ J 

Finally, the feedback control gains in Eq. (4.41) may be expressed in 


partitioned form as 


3(k) s S_(k) j S (k)j . . . j S,(k) 

iJL_ I 1 I • J 


(4.48) 


where S(k) is r x n(J+l) and S^(k) is r x n. From Eq. (4.41) it can be 
seen that S^(k) is the gain multiplying the estimated state vector 
*(k-i|k). Equation (4.41) may nrpw be expressed in-partitioned form as 
u(k) - s (k)j . . . jS.(k) x(k) 

l_°_I_ I J i 

x(k-i) 


x(k-J) 


* l. S . (k)x(k-i) 
i=0 


(4.49) 


This is identical to the result expressed by Eq. (3.106). 
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Eq. (.4.42) may be expressed in. partitioned form,* using definitions 
C4*46), C4.47) and C4.37). 


W 00^ +1 ) * • • * 


M 00 (k+1) . . . M 0J <k+l) 


A(k*M) 0 ... 0 
’0 

• ft 

• • 

*5 

• ft 

• • 

+ 

• 

» 

• 

• 

WjQ(k+l) . . . Wjjik+l)^ 


H J0 (k+l) . . . HjjCkrKO 


o.5 


The partitioned equivalent of this equation is 

r 


W^Ck+l) « 


(kH-1) + A(k+1) i*j«0 


(k+1) 


i,j~0,l,...,J except i~j=0 
0 i,j>J (4.50) 

which is identical to the result-expressed by Eq. (3.108) . 

To examine Eq. (4.43) first express the inverse of Eq. (4.43) in 
partitioned form by dropping the time indices. 

tf'SJtfr 1 r - - r - -1 1.-1-i 


'Therefore Eq, 


S 0 * ‘ * S J 

L_I 



- 



m 

m 

t= 

J' 0 . • • °i 

w oo 

. . . W 0J 

¥ 

+ B 

, , 


• 

ft 

♦ * 

• 

0 

ft 




I 

«5* 

o 

• 

• • • W JJ. 

ft 

0 

» 


• ■ 




• 

c 

[fW^+B]' 1 





) may be written 





0 * • 

. 0, 

w oo • ’ • 

• 

W oj 

• 

4- 6 

0 y l 


* 


• 

ft 

X 0 




• • 

ft 

0 I 




• 

Wjo • • ■ 

W M 

_0 . 


(4.51) 


♦ 


0 10 


+ "oil- • • ! w oo*j-i + w oj! w oo*. 
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’ * ; 2?'V •^S»^ ' r 3»"-C-*~- 


$ 

a 


FI 


When thn time indices are restored to Eq* (4*52) the following results 

.+ * 

ara obtained. 

S t (k) «= [iJj'Ck+l.WHQQCk+Dt (k+l.k) + BCk+1)] -1 

.*'<fcH,k)[W 00 (k+lH l (k+l 1 k) + W 0it+1 (k+l)] (4.53) 

Recall that Wq =* 0 from Eq. (4.J50). Thus Eq. (4.53) is identical to 

® n*. 

m , 

Eq. (3.107). ,Eq. (4.44) may be expressed in partitioned form as 


Mqo • • • M 0J 


k 1 c 

> . . . 0 


• 

o 

o 

• v 

• • 


• 

• 



• 

m • 


i>i 0 

0 


• 

• 

• • 


• 

• 


• 

• 

9 9 


# • 

. 1 


• 

• 

’s’ 

• 

9 

• 

O 

• ^ 


• o 

• - 

-pm 

. . . 0_ 


- w jo • 

• w jj- 

4>o * • • 4>j 

*+ ^ 

to 

o 

• 

• 

* 

J J 


I 0 ... 0 

0 




0 

• 




• • 

• 



• 

• • 

• 




0 . .010 





- ~ 

** 




f 






♦o«00 W 10 «;Vll • • • ^ W 0,J +W 1,J 

n H 00 +W 20 


♦j^oo-Wjd 


*?00 


*?1 


<{>0+^0 • • • ^J +1 ^ S J 

I 0 .... . 0 

0 


♦ * W nT +W TT 

J-l 0J JJ 

I I • 

0X0 
(4.54) 

When Eq. (4.54) is completely expanded and the time indices restored, 
the partitioned result for >L ^(k+1) becomes 

(k+1) = ffl(k+l,k)W 00 (kd-l> + W i+lj0 (k+l)][^(k+l,k) 

+ Kk+l,k)Sj(k)] + ^|(k+l,k)W 0>j+1 (k+l) 


+ W i+l,j+l (k+1) 


(4.55) 
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which, is identical to the result of Eq. (3*109). It should be noted, 

n 

* . i 

of course, that « 0 if i or j > J. . 

* Next expand Eq. (4.39) in partitional form using the following 


definition 


x^ H x(k-i) 

Vk " i£ , x 0 • • • x t, Ko ' • ‘ M 0J 


x Q } + a(k) 


M.a ... Hi 


J J . 

mill £ x!M. .X. } + <x(k) 

i~o j«=o x 13 K 

When the time indices are restored to Eq. (4.56) it becomes 

J J 

V N-k = E{ i S x* (k-i)M. . (k)x(k-j)} + a(k) 
i=0 j=0 13 


(4.56) 


(4.57) 


which is identical to the result obtained in Eq. (3.110) assuming a(k) 
is the same. • * 

Finally, examine ot (k) o^ Eq. (4.40) dropping time indices 

* f\j 'XAj'Xi *\j VWO . ... 

a(k) .If tr(r'wrq) - tr(^'W^SP) + a(k+l) (4.58) 

m. : 

Examine the first*term of Eq. (4.58) in partitioned form 

r- 0 . . . 0, r»f,n • • • %t! r l rQ> r 'V 0*59) 


Wjo * * * WjjJ 0 


Examine the second term of Eq. (4.58) in partitioned form 







\ 
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♦J 1 0 • ■ 
♦ 1 ° 1 


• • 

• • 


.♦j° 


0 

I 

0 


w 


00 


L W J 0 


. w 


0J 


. w 


JJJ 


0 


0 


S 0 * • * S j 

1___ 


p oo * * * P 0J 


p jo* • • P jj. 


♦o 10 

fjoi 


0 

1 


(J) * 0 * • • 0 
» J 


w oo^ 


J W J0*J 


I*0*Q0 + V* 

^X w 00 + ''20^ 


I^ w oo + V* 

*J W 0O 


jSo S 3 P 3 0 


• • • 




1 S j P 1J 
. . J«0 3 3J 


J 


1 J 

• ! * s i p u 

■£1 11 . 


(4.60) 


Let £ denote the ijth n x n submatrix of Eq. (4.60). 

«ij “ I*I w oo + 

Xf"v 

The trace of the total matrix involves only £ .. and 

XJL. 


(4.61) 


trace 1 £.. = trace{ l l [$!W nn + W. +1 0 JipS P.,} 
1^0 i=0 j=0 1 X > 3 3 


(4.62) 


By substituting Eqs. (4.59) and (4.6.2) into (4.58) and restoring the 
time indices, Eq. (4.58) becomes 
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*(k) «c*(k+l) + tr{r f (k+l,k)w(k+l)r(k+l,k)Q(k)} 

* * * 

• J J 

- tr{£ e [a!C k+l,k)W qq( k+i) + \r n (k+l)] 

jssQ j,_Q ** vv X.TX,U 

'■* * 9 

• »J>(k+1, k) S. (k)P (k- j,k-i| k) } (.4.63) 

which, is identical to. Eq. (3.111) . * 

* In summary, then, the solution to the optimal control problem is 

determined in this section using an alternate system representation, 

* ■ **■ .... 

namely, an expanded state representation. The partitioned results are 
identical to those obtained using dynamic programming in Chapter 3 and 
the correspondence is: 


This section 

Chapter 3 

Eq. (4.49) 

Eq. (3.10 6) 

Eq. (4.50) 

♦ 

Eq. (3.108) 

Eq*. (4.53) 

Eq. (3.107) 

Eq. (4.55) 

Eq. (3.109) 

Eq. (4.57) 

'Eq. (3.H0) 

Eq. (4.63) 

Eq. (3.111) 


4.5 Duality o_f Estimation and Control in Systems with Time Delay 
For discrete-time linear systems with no time delay, Kalman 
observed that a "dual" relationship exists between the solution to the 
optimal estimation problem and the solution to the optimal control prob¬ 
lem. His results are presented in the paragraphs below. Section 4.2 
demonstrates that time delay systems may be expressed equivalently in 
an expanded state form. Kalman’s results apply directly to time delay 



systems in this form and the optimal estimation and control solutions to 

such, systems are duals. 

* » 

Kalman's notation is retained for this discussion / since the vari~ 

* * 

■ ■ ■ » . « 

ables in the notation of this paper are not duals, whereas those of Kalman 

are. The relationship between Kalman's notation and that of this paper 
* * 
is also presented for reference. 

Since the partitioned solutions to the estimation and control 

problems yielded significant results, the question arises as to the 

duality of these partitioned solutions. It is demonstrated that the 

* 

principle of duality does not extend to the partitioned solutions of the 

f 

optimal estimation and control problems, 

4.5.1 K alman's Dual Transformation 

Consider the estimation and control problems presented below 

♦ ■ t 

Optimal Estimation Problem . Consider the dynamic system described by 
Eq. (.4.64) and (4*65) 

*(k+l) = + $(k+l,k)u(k) +T(k+l,k)vHk) (4.64) 

z(k+l) » ft(k+lj|f>.+l) + v(k+l) ’ (4.65) 

f ■ , . 

where these equations have all the properties described in Section 4.2. 

Given the observed values z(0) ,z (1) ,... »zf(N-l) find an estimate 

x(k+l|k+l) which minimizes the expected loss 

N-l * „ - 

J N ^ E{ E [x(k+l) - x (k+11 k+1) ] [ x (k+1) - x(k+l|k+l)]'} (4.66) 

k=0 

: i 5 . ... ' 

Optimal. Control Problem . Consider the dynamic system described by Eqs. 
(4.64) and (4.65) where these equations have all the properties described 


* 


• • s t c, t , >1 >i . t: t.. . ta ' 


In Section 4 *2. Given, any state x(k), determine a sequence 

r * * • * 

inCk) juCkdi) >»• • ,u(k+T~l)} of control vectors which minimizes the per¬ 


formance index 


k+T ; 


VIx(K),uCk)] ■= E{ E tx'a)XCi)x(l) + u'Ct-l)B(t-l)u(i-l)]} 

i^k 


(4.67) 


The recursive relations which express the solutions to these 
problems are expressed by Kalman as: 


Estimation Problem 


A*Ct) - Kt+l,t)P*Ct)M’ Ct)[H(t)P*Ct)M f Ct) + RCt)]” 1 (4.68) 


4»*(t+l,t) » <f>(t+l,t) - A*Ct)M(t) 

P*(t+l) « 4>*Ct+i,t>p*Ct>c#> 1 (t+l,t) + QCt) 


tlt Q (4.69) 
(4.70) 


Control Problem 


£*Ct) « [S'COP^CtjMCt) + R(t)} M’(t)P*(t)<{.(t+l,t) 


(4.71) 


♦ Ct+i;t) « 4> (t+i,t) - ma*(c) 

P*(t-1) = J'(t+l,t)P*(t)J*(t+l,t) + Q(t) 


* t - T (4.72) 
(4.73) 


where the correspondence between.Kalman's notation and that of this 
paper is given in Table 4.1. 

„ i 

The principle of duality states that if the form of the solution 

to the optimal estimation problem is known then the form of the solution 

/ 

to the optimal control problem may be obtained by a simple transformation 
on the solution to the optimal estimation problem. The converse is also 
true. Kalman's Duality Theorem may now be stated. * 


Table 4.1 


Correspondence between Kalman Notation and Allgaler Notation 


Estimation 


Kalman 

1 

9 

Allgaler 

M(t) 

$(k+l,k)K(k+l) 

■* 

4*(t+l,t) 

* m 

$ (k+1, k)$ (k+1 1 k+1) P" 1 (k+1 1 k) 

P*(t+1) 

1* (k+11 k) 

• 

M(t) 

* ft < k > 

RCt) 

iUk) 

QCt) 

(k+1 ,k)Q(k)r(k+1 ,k) 


Control 


M'(t) 

i*Ct+l,t) 

P*Ct-'l) 

M(t) 

RCt) 

QCt) 



3>’ (k+l,k)w(k+l)?l(k) 

J«k) 

$(k+l,k) 

B(k> 


X(k) 
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Theorem 4.5. The optimal estimation problem and the 

4 

optical control problem, are duals of each, other in 
the following sense. Let t 0. Replace every mar 
trix F(t) ** F(t 0 +x) in Eqs. <4.68) through (4.70) by 
F 1 (t) = F’CT-t). Then one.has Eqs. (A.71) through 

i 

(.4.73). Conversely, replace every matrix 
F(t) =• F(T-t) in Eqs.' (4.71) through (4.73) by 

F’ (.t) ==■ F*CtQ+T). Then one has Eqs. (.4.68) through 

» 

(4.70) where the quantities described by Eqs. (4.68) 
through (4.73) are presented in Table 4.2. 


The question now arises as to whether the same theorem yields a 
dual result when applied to the subraatrices. Consider, for example, the 

A 

filter gain matrix, A*(t), and the control gain matrix, A*(t), and make 



Table A. 2 


Relationship Between Estimation, and Control Variables 

In Kalman Notation / 

» t 


Estimation 

1. x(t) (unobservable) state vari¬ 
ables of random processes 

2. y(t) observed random variables 

3. tQ first observation 

A. <P (t 0 -H:H-l, tg+x) transition 
matrix 

5. P*(tg+x) covariance of esti¬ 
mation error 

6. A*(tg+ ) weighting of obser¬ 
vation for optimal estimation 

7. <{>*(tp+T+l , tg+i) transition 
matrix for optimal estimation 
error 

8. M(tg+T) observation matrix 

9. Q(tg+x) covariance matrix of 
plant noise 

10. R(tg+T) covariance matrix of 
measurement noise 


Control 

x(t) (observable) state variables 
of plant to be regulated 

« , 

u(t) control variables 
T last control action 
$(T-t+1,T-t) transition matrix 

a" 

P*(T-t) matrix of quadratic form 
for performance index 

£*(T-t) weighting of state for 
optimal control 

$*(T-t+1,T-t) transition matrix 
under optimal regulation 

M(T-t) control transition matrix 

Q(T-t) matrix of quadratic form 
defining state error 

A 

R(T-t) matrix of quadratic form 
defining control effort 
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& . . , &i */ J . ' 


From Eq. (4.68), A*(t) may 


&*(t) 


00 • 


*OJ 


lijo 


« • • 


^JJJ 


be expanded a, a 

i 

_ “I • 


oo: 


l?jo • 7 • 


OJ 


•jjJ 


*0 




'»0 P 00 K 0 + "l" 1 


*0J 


L^jo • • • ‘f’jj.. 


J 


£ 0 yWoo% + R] 


AVWoo H o + R > 

k^O 


-1 


-1 


J J 

* s. 

jt*=0 k ! 


5>VWoo% 


•f r3 


-1 


J J 


«o k y j ^^^ [M ° p °° M o + 


-1 


r r >, 85 * k f 0 <{, i^ i, £k II k tM 0 P 00 M 0 + R] 


-1 


(4.74) 


From Eq. (4.71), A*(t) may be expanded as 


t*Ct} = [M*P 00 M 0 + R]"l ^ 


Im o p oq ! V + Rj “-’ k z 0 K\o\ 


i 


00 


• • • 


0J 


L p jo • • • p jjj 


^00 


• • • 


I J 
» 2 


* * • - ^ P kJ 

j k-0 K ^ 


00 


<t> 


JO 


^OJ 


l?J0 • * 

. . . 4> 


* 


JJJ 


0J 


JJ 



* rM* 


a-.... .. - 
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- woo»o * »■' ^ jj.’ty» 


IN o p oo"o*'J ' 1 <W» 


i-i / 


j j 


or. • A*Ct) - 1%P 00 M 0 + W-* 


(4.75) 


Application of Theorem 4.5 to Eq. C4.74) yields 


J J 


• ^i Ct)] ' “ k =o^ p ik^ [M i p oo M o + R'l 


11 ““1 


or, 


^Ct) 


J J -1 

Z Z [MiP„„M. + R] XP,J. 


£«0 k~0 


(TOO 0 




(4.76) 


From comparison of Eq. (4.75) and C4.76) it is obvious that Theorem 4.5 

can not. in general, be extended to the submatrix results. Since, in 

general, this is not true, then it can be concluded that the duality 

expressed by Theorem 4.5 does not apply to the corresponding submatrices. 

Attempts by the author to modify the duality theorem such that the re- 
» * 

suits may be extended to the submatrices have not been fruitful. If such 
e relationship c~.ild be demonstrated then the results of Chapters 2 and 
3 should be the duals of each other. 


4.6 Summary and Conclusions 

In tills chapter an alternate expression for discrete systems 
with time delay has been developed. This expanded state representation 
is not new, yet it is shown to yield the same results as Chapter 2 and 


which used a different approach. These results are, then, submatrix 
solutions to the. expanded state representation. This method presents 
* substantial savings in computation time as well as storage, Examples 

* 

of these reductions are presented in Chapter 5. 

Finally* the expanded st^te representation allowed demonstration 

4■ 

of duality for time delay systems, although this result could not be ax- 

« . 

* 

tended to the corresponding submatrices. * * 




• CHAPTER V 


■f 

COMPUTATIONAL ADVANTAGES OF PARTITIONED SOLUTIONS 

* 

TO OPTIMAL ESTIMATION AND CONTROL PROBLEMS 

/ 

* * * 

5.1 In. reduction 

Chapter 4 demonstrates that the filter and control gains obtained 
in this dissertation are Identical to results published many years ear- 
Her * using an expanded state representation of time delay systems 

The principal difference is that a partitioned form of the resulting al¬ 
gorithms are used rather than the expanded matrices. Since the expanded 
* 

state representation is generally discredited because of the resulting 

» 

extensive computation required, one might ask whether or not the parti¬ 
tioned results represent a significant computational savings. 

Vj t 

In this chapter analytical expressions are developed for the 

* ■ 

number of multiplicative and additive operations required for digital 
computer solution. Options are presented which reduce the number of op- 

• • t * 

erations even further in the partitioned form. These are at the expense 
of additional software requirements, however, and in some cases this more 
than offsets the advantages of the partitioned form. This is generally 

'4 

true where the system dimension and time delay are small. The savings 
in computation time are strong functions of the system order, n, and the 

t 

time delay magnitude, J. No further attempt is made here to define the 
additional software requirements for the partitioned solution since that 
depends to a great extent on the ingenuity of the programmer. 



Ill 


Some illustrative examples are presented, where computation time 

Is reduced by more-than 90% and storage by more than 60%. For a unit 

* 

* a 

delay, a 30% reduction in both, storage requirements and computation time 
is typical. 

/ . 

5.2 Computational Requirements lor Estimation Algorithms 

In this section a method of determining the number of discrete 
multiplicative and additive operations is presented. This method is 
then used to develop analytic expressions for the number of required 

operations in terms of the dimensions of the system matrices. Examples 

' » 

are presented at the conclusion of this section which demonstrate a sig- 

* ' » 

nificant reduction in computational requirements for the partitioned al- ‘ 
gorithm when compared with the entire matrix algorithm of the expanded 
state form. 

Consider the multiplication of two matrices A and ,E of dimensions 
• * 

i x m and m x n respectively. The number of discrete multiplicative op¬ 
erations is imn. The number of discrete additive operations irequired is 
Umrl) n. For ease in calculations, it is assumed for the remainder of 
this chapter that the number of required additive operations is Iran. 

The number of required multiplicative and additive operations 
is determined for the partitioned algorithm expressed by Eqs. (2.115), 
(2.116), (2.117) and (2.118). The dimensions of the matrices involved 
are: P(k+l,k+ljk) - n x n, ^(k+ljk) - n x n, P(k-i,k-j|k) - n x n, 
P(k+l~i,k+l|k) - n x n, f(k+l,k) n x p* Q(.k) - p x p, H(k+1) - m x n, 

- R(k+1) - m x m, K^(k) - n x m. 
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4 4 


J?(kKL,k.+l|k) * t i £ <f>,(k+1,Ic)P(k-i,M’| k)$ J (k+1,k) 

1=0 ;)=0 ^ 

+ l)(k+l,k)Q(k)r'(k+l,k) 


(5.1) 


♦ ' 

The product 4 ^(k+1,k)P(k-i,k-j|k) requires multiplicative operations. 

This product multiplied times <f>'(k+l,k) requires an additional n ” 5 multi- 

, 4 

plicative operations, resulting in 2n J multiplicative operations to form 
the product 

f^Ck+l, k) P Ck-t, k-j 1k) <f ’ (k+1, k) ' 

The double sum means there are (J+l) 4 such, terms, or, a total of 
2n^fj+l) 2 multiplications in the double sum. Similarly, the second term 
requires a total of n 2 p + p 2 n operations, yielding a total of 
2n^(J+l) A + n 2 p + p 2 n multiplicative operations. The number of additive 
operations are n 2 [(j+l) 2 - 1] for the double sum, plus n 2 by adding 
rCk+l,k)Q(k)p*(k+l,k) to the double sum, or a net of n 2 (J+l) 2 . To this 
must be added the number of additive operations due to the multiplica¬ 
tion process or 2n^(J+l) 2 4* n 2 p + p^n such additions. The total number 
of operations required in determining P(.k+l,k+l|k). are 
multiplications: 2n^(J+l)^ + n 2 p + p 2 n 
additions: 2n^(J+l)^ + n^p + p 2 n + n^(J+l)^ 

A similar analysis yields the following results for the remaining equa- 

» 

tions of the partitioned estimation algorithm. 

K^k+1) = P(k+l-i,k+11k)H’ (k+1) [H(k+1)P(k+1, k+11 k) H 1 (k+1) (5.2) 


+ R(k+1] 


1 = 0,1, ... , j 


(5.2) 





Total Number of Operations for 1-0,1, ... , J 
multiplications: n 2 mtJ+2] Hb 2nm 2 
additions: n 2 mlJ+2] + 2nm 2 + m 2 

J ~) 

PCk-j,k+l|k) - 1 P(k-j ,k-i|k)<|) * (k+1,k) j 

1=0 * * 


/ 

9 


0 , 1 , 


• • • »i 


J 


(5.3) 


Total Number of Operations for j =0, 1, ... , J 

* multiplications: n3j(J+l) 

* 

additions; n 3 J(J+l) + n 2 J 2 

J?(k-i,k-j |k) = P(k-i,k-j|k-l) - (k)H(k)P(k,k-j [k-1) (5.4) 

Total Number of Operations for i,j =0, 1, ... , J 
multiplications: n 2 m[(J+l) 2 + (J+l)] 
additions: n 2 m[(J+l) 2 + (J+1)] + n 2 (J+l) 2 
The total number of' operations required for one complete iteration of 
the partitioned estimations algorithm are .given in Table 5.1. These 
results are referred to as PARTITION E-l. 

The number of required computations may be further reduced by 
recognizing that not all of the delayed states affect explicitly the 
one step transition. That is, some of,the operations are not required 
since some of the ^(k+ljk) = 0. Let j be the number of ^(k+l.k) which 
are not identically zero* Then it can be shown that the number of .com¬ 
putational operations becomes as shown in Table 5.1. Hereafter, these 
results are referred to as PARTITION E-2. 

Similarly, it can be shown, for the expanded algorithm given 
by Eqs. (5.5), (5.6)’and (5,7) that the number of required operations 




Table: 5.1 


Analytic Expression, of Number of Computational Operations 

Required for Various Estimation Algorithms 
# * 

/ 

0 

PARTITION E-l , 

multiplications; n 3 [2(J+l) 2 + j(j+l)] + n 2 m[(J+l) 2 + 2J + 3] 

* »■ . ‘ 

+ 2m 2 n + n 2 p 4- p 2 n 

additions; n 3 [2(J+l) 2 + J(J+10] + n 2 m[(J+l) 2 + 2J + 3] 

+ 2m 2 n + n 2 p + p 2 n + m 2 + n 2 [2(J+l) 2 + J 2 ] 

j * 

PARTITION E-2 

• ■■ — ■■■ i i« ..I i . 11, i i . —II.. 

multiplications; n 3 [2j 2 + Jj] * n 2 m[2J + 3 + (J+l) 2 ] + 2m 2 n + n 2 p + 
additions; n 3 [2j 2 + Jj] + n 2 ra[2J + 3 + (J+l) 2 ] + 2m 2 n + n 2 p + p 2 n 

+ n 2 [j 2 + J(j-l) + (J+l) 2 ] + m 2 

EXPANDED 

multiplications: 2n 3 (J+l) 3 + 4n 2 m(J+l) 2 + 2m 2 n 

• * » > * 

additions: 2n 2 (J+l) 2 + r 2 + 2n 3 (j+l) 3 +4n 2 m(J+l) 2 + 2m 2 n 


* a 
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# 

Is as shown in Table 5,1. The piatrix dimensions are given in Table 5.3. 

• * t » « .* 

fc(k+l) ** $(kil1k)?!' (k+1)[ft(k+l)?(k+1|k)ft(k+l) + feCk+DT 1 (5.5) 

, 

if (k+1 |k) ^ ^ (k+1 I k)$ (k| k) c|> * (k+1, k) + ^(k+1,k) Q(k) r' (k+1,k) (5.6) 

► * * t 

J(k+l|k+l) ~ [I - & (k+1) ft. (k+1) J $ (k+1 \ k) (5.7) 

The effect of system order-(n) and time delay magnitude (J) on 
the required computational effort can be seen from. Table 5.2^where a 
number of different examples are considered. The data for PARTITION E-2 

m 

assumes,a single delay (j - 2) in the system, where the delay magnitude 

• 

is still specified by J. Note for the cases where there is just a unit 
delay (n - 2, J ** 1) or (n - 10, J - 1) that the partitioned computations 
require only 60% of that required by the expanded form. For multiple 

, i » ' * 

delays, where the delay is longer (n ~ 1, J - 19) the computational re- 
quirements are less than 10% of those required by the expanded form. 

In general, the savings achieved are an increasing function of the time 
delay magnitude. 

5.3 Storage Requirements for Various Estimation Algorithms 

The storage requirements' are specified by the size of the ma- 

* * 

trices used in the algorithms. The partitioned algorithm is given by 
Eqs. (5.1) - (5.4) and the expanded state algorithm is given by Eqs. 

(5.5) - (5.7). The matrix dimensions and required storage are given in 
Table 5.3. Recall that PARTITION E-2 requires storage only for those 
values of ^(k+l,k) which are not null matrices. Storage requirements 
are computed for a number of examples in Table 5.4. A 30% reduction in 
required storage is achieved for a unit delay (J - 1) and a maximum 
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Table*5.2 


Examples oi; Required Number op Computational Operations 
For Various Estimation Algorithms 


n 

J 

m 

• P 

operation 

EXPANDED 

PARTITION 

E-l’PARTITION 

1 

3 

1 

1 

mult. 

200 

\ 

73 

45 





add. 

233 

115 

69 

2 

1 

1 

1 

mult. 

*200 

126 

126 





add. 

233 

* . 163 

163 

4 

0 

1 

1 

mult. 

200 

200 

>■ 

200 





add. 

233 

233 

’ 233 

1 

19 

« 

1 

1 

mult. 

18060 

1625 

491 





add. 

18861 

2786 

915 

2 

9 

1 

1 

mult. 

18060 

2812 

702 




* 

add. 

18861 

3937 

1155 

4 

* 

4 

1 

1 

mult. 

18060 

5084 

1628 





add. 

18861 

6140 

2157 

10 

1 

1 

1 

mult. 

18060 

11030 

11030 





add. 

18861 

11931 

11931 

7 

5 

1 

1 

mult. 

157087 

37457 

13790 





* add. 

161616 

* 41890 

16486 
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savings of 60% can be seen from the analytical expressions for storage 

* 

requirements in Table 5,3. As In the preceding section, storage re- 
> qulreraent savings Increase as the time delay magnitude Increases. 

t 

5.A Computational Requirements for Control Algorithms 

\ 

The methods of Section 5.2 are*used In this section to deter¬ 
mine the computational requirements for the partitioned and expanded 
control algorithms. Results are obtaine;! for the partitioned algorithm 

under three conditions. First it is examined for the general case with 

* * 

all <{>£ not equal to zero. Next the computation is reduced by eliminating 

* 

all operations for which <{>^ is equal to zero. Finally, it is shown that, 
at a nominal cost in additional storage requirements, a substantial re- 
duction in computation time may be achieved by storing the results of 

some operations which are repeated. As in the estimation problem, the 

• • ♦ 

partitioned algorithm represents a computational savings of 30% to 90% 

." ' “ 

* . * ... 

* 

over the expanded form. * 

* 

PA RTITION C~l : The partition algorithm is expressed by Eqs. 

(5.8) and (5.9). 

S(k) « [ifi 1 (k+l,k)W 00 (k+l)if (k+l,k) + B<k)] -:L ^' (k+l,k) 

•. • .[W 00 (k+l)4, i (kH,k) + H 0ii+1 (k+l)) ' (5.8) 

W(k) *= [<i, i (k+l,k)W 00 (k+l) + W 1+1>0 (k+l))[ l (. ;j (k+l,k) + f(k+l,k)S,(k)] 

+ <f£(k+l,k)U Q( j +1 (k+l) + H 1+1>j+1 (k+l) + A(k) 

(5.9) 

The matrix dimensions are given in Table 5.7. Application of the methods 
of Section 5.2 yield the number of required operations given in Table 5.5. 
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Table 5.4 

* 

Examples of Required Storage 
For Various Estimation. Algorithms 


n 

J 

m 

P 

EXPANDED 

* 

PARTITION E“1 

•'PARTITION E-2 

1 

3 

1 

1 

* 

62 

32 

30 

2 

1 

1 

1 

62 

44 

42 

4 

0 

1 

1 

62 

* .62 

62 

1 

* 

19 

1 

1 

1262 

464 

♦ 

446 

2 

9 

1 

1 

1262 

506 

474 

4 

4 

1 

1 

1262 

590 

542 

10 

1 

1 

1 

1262 

842 

842 

7 

5 

1 

1 

* 

5420 

2410 

2263 

(j«3) 


i 



PARTITION C-2 : II the operations are eliminated lor those 
4>^Ck+l,k) which are equal to zero, the number of operations is reduced. 

If the number of $^(h+l,k) not equal to zero is j, the resulting number 

of computations can be shown to be those given in Table 5.5. 

* 

PARTITION C-3 : Examination of Eqs,. (5.8) end (5.9) reveals that 

* 

the term n-fQQ(k+l)<j>^(k+l,k) + Wq i+i^+l)] occurs in the first equation 

and its transpose occurs in Eq. (5.9). This term may be stored at a 

* 

storage cost of n^(J+l) locations, these eliminating the necessity of 
re-computing a second time for each, iteration. The resultant computation 
requirements are given in Table 5.5. 

EXPANDED i The algorithm for the expanded form of control solu¬ 
tion is expressed by Eqs. (5.10) and (5.11). 

$(k) « [$' (k+1,k)(k+1)$(k+1,k) + S(k),f' 1 ^ t Ck+l,k)W(k+l)$(k.-l*l,k) 

* 

• •: (5.10) 

W(k) *= $'(k+l,k)W(k)[?(k+l,k) + ^<k+l,k>S<k)] + A(k) (5.11) 

The matrix dimensions are given in Table 5.7. The required number of 

» * 
operations per iteration are given in Table 5.5. 

The effect of system order (n) and time delay magnitude (J) on 

the required computational .effort can be seen in Table 5.6 where a hum- • 

> 

ber of different examples are considered. The data for PARTITION C-2 
and PARTITION C-3 assumes a single delay (j - 2) in the system where 
the delay magnitude is still expressed by J. For the cases where there 
is just a unit delay, (n =2, J * 1) and (n = 10, j = 1), the PARTITION 
C-3 computations are only about 30% of those required for the expanded 




a 



'1 
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Table 5.5 

i 

• * 
Analytic Expressions for the Number of Computational Operations 
Required for Various Control Algorithms / 

+ 

PARTITION C-l 

♦ " 

multiplicatio ns: n 2 [2(J+l) 2 + J 4* 1] Hh n 2 r[l + (J+l) + (J+l) 2 ] + 2nr 2 

additions: n 2 [2(J+l) 2 + J + 1] + n 2 r[l + J + 1 + (J+l) 2 ] + 

• JllTTMll-UII II —l.-r . j. 

+ 2nr 2 + r 2 + n 2 [J 2 + J + 2 (J+l) 2 ] 

* 

PARTITION C-2 

multiplications : n 3 [(J+l) 2 + j(J+2)J + n 2 r[l + <J+1) + (J+l) 2 ] + 2nr 2 
additions : n 2 [ (J+l) 2 + j(J+2)] + n 2 r [1 + (J+l) + (J+l) 2 ] + 2nr 2 

+ r 2 + n 2 [J 2 + 2Jj + 4j -J -2] 

PARTITION 03 

multiplications ; n 2 [(J+l) 2 + j(J+l)] + n 2 r[2J+3] + 2r 2 n 

additions ; n 3 [(J+l) 2 + j(J+l)] + n 2 r[2J+3] + ‘2r 2 n + r 2 | 

+ n 2 1J 2 + jJ + 2j] ‘ 

• ■' V 

additional storage required : n 2 (J+i) 

EXPANDED 


multiplications : 3n 2 (J+l) 2 + 3n 2 r(J+l) 2 + 2n(J+l)r 

additions: 2n 2 (J+l) 2 + r 2 *+ 3n 3 (J+l) 3 + 3n 2 r(J+l) 2 + 2n(j+l)r 
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Table 5.6 


Comparison o£ Required Number of; Computational Operations 
For Various Control Algorithms 








j*=2 

j=2 


N 

J 

r 

operation 

EXPANDED 

PARTITION PARTITION PARTITION 

EXTRA 






C-l 

C-2 

C-3 

STORAGE 

1 

3 

1 

mult. 

248' 

59 

47 

‘ 35 

4 




add. 

281 

103 

71 

» * 

55 


2 

1 

1 

mult. 

' 248 

112 

112 

88 

8 

m 



add. 

281 

153 

153 

117 


4 

0 

1 

mult. 

248 

248 

248 

184 

16 



\ 

add. 

281 

281 

281 

217 


1 

19 

1 

mult. 

25240 

1243 

863 

449 

20 




add. 

26041 

2420 

1288 * 

853 


2 

9 

1 

mult. 

25240 

2048 

1424 

1048 

40 




add. 

26041 

3209 

1881 

1453 


4 

4 

1 

mult. 

25240 

4016 

2872 

. 2424 

80 




add. 

26041 

5137 

3417 

2912 


10 

1 


/ mult. 

25240 

10720 

10720 

8520 

200 




add. 

26041 

11721 

11721 

9221 








j=3 

3=3 


7 

5 

i 

mult. 

227640 

28875 

20228 

19183 

294 




add. 

231169 

33874 

23169 

21437 



form. For multiple delays, where the delay is longer (n « 1, J « 19) 
the computational requirements of PARTITION C-l are*less than 5% of those 
required by the expanded form, in general, the savings achieved are an 
increasing function of the time delay magnitude. 

5.5 Storage Requirements for Various Control Algorithms 

The storage requirements are specified by the size of the ma- 

* 

trices used in the algorithms. The partitioned algorithm is given by 
Eqs. (5.8) and (5.9) and the expanded state algorithm is given by Eqs. 
(5.10 and (5.11). The matrix dimensions and required storage are given 
in Table 5.7. Recall that PARTITION C^2 requires storage only for those 
values of <{>.(k+l,k) which are not null matrices. PARTITION C-3 requires . 
n^(J+l) storage in addition to that of PARTITION C-2 as explained in the 

preceding section. Storage requirements are computed for a number of 

> * * 

examples in. Table 5,8. A 30% reduction in required storage is achieved 

, v 

for a unit delay (J- 1) and a maximum savings' of 60% can be postulated 

\ 

from the analytical expressions for storage requirements in’ Table 5.7. 

Storage requirements decrease as the time delay magnitude increases. 

* * 

• 5.6 Summary 

As. demonstrated in,the preceding sections the partitioned algor¬ 
ithms obtained in this dissertation represent a 30% - 60% reduction in 
storage requirements when compared with, the expanded state representa¬ 
tion. A reduction in computational effort of 30% - 95% is also demon¬ 
strated. Even greater savings can be realized depending on the example 
selected. Both the reduction In computation time and storage make the 
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Table 5.8 

* 

Examples of Required Storage 
for Various Control Algorithms 


y 


n 

J 

r 

EXPANDED 

PARTITION C-l 

PARTITION C-2 (j*2) 

1 

3 

1 

57 

26 

25 

2 

* 1 

1 

57 

35 

35 

4 

0 

1 

57 

57’ 

57 

1 

19 

1 

1241 

441 . 

425 

2 

9 

1 

1241 

449 

435 

4 

4 

1 

1241 

505 

473 

10 

1 

1 

1241 

731 

731 

7 

5 

1 

, 5377 

2010 

1961 (j=3) 




partitioned solutions quite attractive. In retrospect, the expanded 
state representation of time delay* systems does provide algorithms which 
, are computationally feasible after all. The need for a method to solve 

J 

r 

the expanded equations efficiently is resolved by this paper as the ex¬ 
amples of this chapter clearly demonstrate. It should be recalled, how¬ 
ever, that the partitioned solution has additional software requirements 
and this may offset the postulated advantages when J is small. 




CHAPTER Vr. 


CONCLUSIONS AND FUTURE WORK 

* * * 

6.1 Conclusions 

In Chapters 2, 3 and 4, algorithms are obtained for determining 

» * 

the optimal filter and optimal control gains fot discrete linear systems 
« * 

with, time delay. In Chapter 4, it is demonstrated that these results 

• 

are actually partitioned solutions to the expanded state representation 
of such systems. This expanded representation of time delay systems has 
been, studied but deemed computationally unacceptable by previous authors. 
The unique nature of such a representation for time delay systems, how¬ 
ever, yields a substantial reduction in the computational requirements 
when the solution is expressed in partitioned form. It is not unlikely 
that the computations resulting from the partitioned solutions be 80% 
to 90% less than that formerly required by the expanded matrix solution. 
Many problems previously considered computationally unfeasible can as a 
consequence, now be solved. 

6.2 Future Work 

In the course of achieving these results, a number of related 
problems associated with time delay systems have arisen. 

1. What are the conditions under which continuous stochastic 
systems may be expressed as discrete systems? The initial conditions 
on the delay, since it is continuous, and the problem of delays which 
are not integral multiples of the sample period cause difficulty here. 
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Soc\e preliminary vjork by the author suggests tha,t this may be possible 

* * 

for serial systems. 

2. Can controllability and observability as expressed in Appen- 

• t 

dix A, be expressed in partitioned form? The achievement of significant 
results by partitioning the expanded representation of time delay systems 

suggests that this approach may be extended to other aspects of time de- 

. . * . 

lay systems. Xn Chapter 5 this effort failed in examining duality, how- 
*• 

ever, so there are some limitations on extending this approach. 

* • 

3. Are the results of this paper computationally superior to a 

high order approximation of the time delay? Since the required computa¬ 
tion increases as a significant function of the time delay magnitude, 
it may be more economical (and just as precise) to use a high order ap¬ 
proximation to the delay and then use standard techniques such as the 
Kalman filter for obtaining the final result. 

■4. .Can continuous algorithms be obtained by examining the lim¬ 
iting case of the discrete solutions? The continuous estimation and con- 

3 8 

trol solutions are known * for systems with time delay, but they are 
computationally untractable for high order systems (greater than n - 2) 
or more than a single unit delay Perhaps the solution to these contin- . 
uous-time equations may be determined by examining the limiting case of 
the discrete-time equations. This is certainly true for systems with no 
time delays. ; 
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APPENDIX A 

■ 

REQUIRED PROOFS FOR CHAPTER II 



REQUIRED * PROOFS FOR CHAPTER II 


In this appendix necessary* and sufficient conditions are devel¬ 
oped for the observability and controllability of discrete linear systems 

* * 

With time delay. The development below: is for the expanded state repre¬ 
sentation of time delay systems. The reader is referred to Section 4.2 
for definitions of the matrices and vectors of* the system and measures 


ment equations (A.l) and (A.2). 

» * f 

x(k+l) « (k.H-1 ,k)x(k) + $Ck+l,k)uCk) 

* *■ 

, ^fk+l) B ft(k+l)Sc(k+l) 


CA.l) 
(A. 2) 


Observability ; First, an observable system is defined as follows: 

Definition : The discrete linear ‘system of Eqs. (A.l) and 

* 

CA.2) is observable,if x(0) can be determined from the 

. set of measurements’ $(1), z(N)} for some finite 

N. If this is true for any initial time (k « 0 corres- 

* 

ponds to tg), the system is completely observable. 


The following theorem can now be proved. 

Theorem A.l : The discrete*linear system of Eqs. (A.l) 

* 

and (A.2) is completely observable if and only if the 
mN x n(J-fl) matrix 


fl(l)6(l,0) 


3t(n,n-i) . . . la . o ) 


(A. 3) 
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is of rank n(J+l) for some N ? 0. 


Proof: It ts sufficient to consider the case where u(k) »,0. 

* * / 

5cCk+l) * $(k+X,k)x(k) 

» * # 

z(k+l) » ft(k-KL)xCk+l) 

► 

* 

k ■* 0, 1, ... since fo(k) is assumed known for all k. 


(A.4) 

,CA.5) 


Consider the sequence of measurements {z(l), ... , z(N)}. , In 
order for the matrix defined by Eq. (A.3) to be of rank n(J+l), N is 
chosen such that the inequality mN £ n(j+l) is satisfied. 

• From Eqs. (A.4) and (A.5) the following equations result from 

the sequence of measurements. 

*<l) » Sa)x(i) = ki)4(i,0)x(0) 

2<2) = %(2)i(2) = fl(2)9 C2|l)$(l,0)x(0) 


z(N) * HCN)x(N) *= tt|N)^(N,N-l) . . . 4(l,0)xC0) 
These equations may, be written In matrix form as 


z N e 

where z^ ^z(l) 


CA.6) 


zbo 


% 3 ft(i)|ci,o) 

kC2)$(2,l)$Cl,0) 


ftCN)$(N,N-l) . . . $(1,0) 


(A.7) 


It’is clear that•z N is an mN vector and x(0) is an n(J+l) vector. Since 
mN > n(J+l) and x(0) is arbitrary, the theory of linear equations may 
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now be, Invoked. This theory states that there exists a unique solution 
to Eq. (A«6) if and only if the matrix as expressed by Eq. (A-7), is 
of rank n0+1)» Thus Theorem A. 1 is established. / 


Controllability ; First, controllability is defined in the following 

* 

sense. 


1. Definition A. 2 : The discrete linear system of 
Eq. (A.l) is controllable at time k « 0 (corres-^ 
ponding to an initial time tQ) if there exists 

i » 

a control sequence $CO),uCl)»... >ft(N-i)} depend- 

* 

ing on &(0) and the initial time, for which &(N) 
may be selected arbitrarily * where N is finite. 

If this is true for all x(0) and initial times, 
the system is completely controllable. 


The following theorem can now be proved. 

Theorem A.2 ; The discrete linear system of 
Eq. (A.l) is completely controllable if and only 
if the n(J+l) x rN matrix 

H(N,N-1)...<J>(2,1)<K1»0) .... f(N,N-l)^(N~l,N-2) ^(N,N-l)j 

(A*. 8) 


is of rank n(j+l) for some N > 0 


Proof ; The relationships for x(k) may be expressed in terms of x(C) and 
the control from Eq. (A.l) as 



















-• * 


* 


*■ 
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xCi) ** «>d,o)xCo) t ?(i,o)uCo) 

*C2) * fC^i)?a,o)xco) + ?c2.D?a,o)uco) + ?c2 # i)Sci) 


• » » 

, x*(N) ■'^(NjN-l) . . . $(l,0)xCO) 

t 2 

Hi* . . 

+ z i n 4.(j,^i)jii;a,i-i)uCi-i) 

• i»l j=l 

* * 

This sequence may be expressed in matrix form as 


3!GO - ? N S C0> * yi N . ' (A. 9) 

*- 

where x(N) is arbitrary 

* » » 

?„ S^CN,N-1) . . . ?(2.1)?a,0) ' (A. 10) 

? N =4>0< ! N-1)...<K2,1)i|i(1,0) ... '*(H,N-1)i|i(N-1,N-2) iKN,N-1) 

(A.l ) 

*V 'O.*, 

U N 2u(0) 

• ft 

. ■ ■ ■ • . ' • ' . 

# 

u(N-l) (A.12) 

The definition requires that x(N) be arbitrary and it can be seen that 
x(N) is an n(J-KL) vector and is an rN vector where rN > n(J-fl). 

Once again, Eq. (A.9) has a unique solution, if and only if the matrix 
defined by (A.11) is of rank n(J+l) and Theorem A.2 is established. 





, APPENDIX B 

OBSERVABILITY AND CONTROLLABILITY OF TIME DELAY SYSTEMS 

* » » 

* f 





OBSERVABILITY AND CONTROLLABILITY OF TIME DELAY SYSTEMS 

/ 

* * 

f m 

k * 

in this appendix., properties (2.39) - (2.47) ara established. 
Recall the system and measurement equations for systems with, time delay. 

J 

x(k+l) * t <f> 1 (k+l,k)x(k-i) + r(k+1 ,k)w(k) + ip(k+1,k)u(k) (B.l) 
i«=0 

z(jk+1) « H.(k+l)x(k+l) + v(k+1) * (B.2) 

These equations may also be imbedded in an expanded state representation 
x(k+l) ~ <jT(k+l,k)x(k) + t^(k+l,k)w(k) + $(k+l,k)u(k) (B.3) 

’ z(k+l) « ?t(k+l)x(k+l) + v(k+l) (B.4) 

where the reader is referred to Section 4.2 for definitions of the ex¬ 
panded matrices and vectors. 

The following properties of the vectors are recalled for later 
use in the proofs below. 

{x(-i), i«0,l,.... ,J) is a zero mean gaussian random n-vector 

(B.5) 

(w(k), k=0,l,...} is a zero-mean, p-dimensional gaussian white 
sequence (B.6) 

(v(k) , k=l ,2,...) is a zero-mean, m-dimensional gaussian white 
sequence .... (B.7) 

{v(j), j-l,2,and {w(k), k-0,1,...} are independent (B.8) 
{x(-i), i-0,l,,..,J} is independent of fv(k+l), k=0,l,...} and 

( 

{w(k), k=0,1,...}. (B.9) 

(u(k), k=0,l,...} is known or may be specified as desired (B.10) 


it can be.shown that for a, system described by Eq. (B.3), the 

> t * » * 

state of; the' system at time k may be expressed in terms of the initial 

t * 

state, noise disturbance and control vector as / 


5c(k) ** $(k*0)x(0) + Z $ (k,£)r (fc ,£-l)w(i-l) 


+. r ^(k,«)5;a,ii-i)wa-i) cb.h 

A*=l 

0 

The partitioned expression for x(k) can then be written as 

J J k - 

x(Jk) « Z $Qi(k,0)x(-i) + z z $ 10 Ck,i)rU,X-l)w(i-l) 

i **0 i =0 £=1 


J k. 

+ Z £ <J>i 0 CM)iK^-l)uU-l) (B.12 

i-0 £<L 

* * 

where <J»^jCk,£) is the ijth. n x n submatrix of $(k,£). With the above 
results, the .properties of Chapter 2 can now be established. 


Property (2,39) : The stochastic processes {x(k), k=0,l,...} and 
{z(i), i"l,2,...,j} are gaussian’. 


(B. 13 


Proof : Consider Eq. (B.12). Recall that (u(j), j=0,l,...} is a deter¬ 
ministic quantity by property (B.10), -Since {x(-i), i=0,l,...,J} and 
each (w(£-l), £=1,2,..„,k). are gaussian by hypothesis, it follows that 

x(k) is also gaussian for k - 0, 1, ... , since it is merely the sum of 

• » - * 

gaussian random vectors plus a deterministic vector. Consequently, for 
any integer m and set of time points {t^, ..., t m el} the set of ran 

dom n vectors x(ti), x(t 9 ), ..., x(t ) is jointly gaussian distributed 
and the assertion is proved. ^'' 



Property (2.4Q): E£x(j )w’ (k)} *=* 0 for all k^4, j « 0* 1, ... (B.14) 

» .* • * *«.'« 
t * * »♦ + * 

Proof : Eq. (JB.12) may be substituted ijato Eq'. (B.14) to yield 

* 

J* * . • . " ■ 

E{x(j)w* (k)} « £ ♦oi^OWxC-Dw 1 (k)} 

i c 0 

+ b b <t> 1Q (j,i)ra,i-i)E{wa-i)w f (k)> 
i-O i«i 

j j 

• + £ £ ^nCj (i »i"'l)u(A“l)E (w* (k)} (B.15) 

# i«o i=i 

From property (B.9) the first term on the right-hand side of Eq. (B.15) 
vanishes for all k « 0, 1, ... . Since’{w(k), k-0,1,...} is a white se¬ 
quence. it follows that E Iw(a~1)w* (k)} *? 0 for all k ^ SL- 1* Since 
J5 - 1 , 2, ..., j it is clear that the second term vanishes for all 
k > j-1 or k i. j. The third term vanishes since {w(k) , k=0,l,...} has 

zero mean. Hence, Property (B.14) is established. 

. • 

Property (2.41) : E{z(j)w’(k)} = 0 for all* k ;> j , j « 0, 1, ... (B.16) 

Proof : Substitute Eq. (B.2) into Eq. (B.16) 

E|z(j)w , (k)} * H(j)E*[x (j)w* (k)} + E{v(j)w'(k)> (B.17) 

By virtue of Eq. (B.14) the first term on the right-hand side of Eq. 
(B.17) vanishes for all k £ j, j ~ 0, 1, ... . The second term vanishes 
by property (B.8) and the assertion (B.16) is established. 
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Property CM2) ; Efx(4)v’ (k)} - 0 for all J * 0, 1, ... and k « 1, 2, ... 

‘ * * * , • 

(B.18) 


Proof ; Substitute Eq, (B.12) into Eq. (B.16). 

J' 

Elx(j)v’Ck)} » J J 01 (j,0)E{x(-i)v'(k)} 

i*=0 


+ e e ^ n (.j^)ra,i-i)E{ W a-i)v'(k)} 

i«0 £*=1 


+ E E $ (B.19) 

i~0 £=1 1U< * 


The first term on the right-hand side vanishes by property (B.9). The 
second and third terms vanish by properties (B-8) and (B.7) respectively, 
and property CB.17) is established. 


Property (2.A3) : E{z(j)v'(k)} = 0 for all k > j where j,k = 1, 2, ... 

(B.20) 

Proof ; Substitute Eq. (B.2) into Eq. (B.18) 

Eiz(j)v'Ck)} - H (j ) E {x (j ) v * (k) } + E{v(j)v’(k)} (£.21) 

The first term on the right-hand side of Eq. (B.19) vanishes by property 
(B*17). The second term is equal to zero except for j ~ k since 
{v (k) , k=l, 2,.. .} is a white sequence.* But k > j by hypothesis and the 
assertion (8.20) is eatablished. 








T 



REFERENCES CITED 

' * . . 

/ 

f 

*» * , .. it. 

* » 

1. Priemer, R., and A. G. Vacroux. "Estimation in. Linear Discrete Sys¬ 
tems with. Multiple Time Delays," IEEE Transactions on Automatic 
Control (Short Papers), Vol. AC-14, pp. 384-387, August 1969. 

2. Kalman, R. E., and R. S. Bucy. "New Results in Linear Filtering and 
Prediction Theory," Journal of Basic Engineering , Trans. ASME, 

Vol. 83, pp. 95-108, March 1961. 

3. Kwakernaak, H. "Optimal Filtering in Linear Systems with Time De¬ 

lays," TEEE Transactions on Automatic Control., Vol. AC-12, pp, 169- 
173, April 1967. * ' 

4. Larson, R», and D. R. Wells. "Combined Optimum Control and Estima¬ 
tion of Serial Systems with Time Delay P " Proc . JACC , pp. 23-33, 

1969. 

5. Meditch, J. S. Stochastic Optimal Linear Estimation and C ontrol . 

New York: McGraw-Hill, 1969. * 

6. Williams, T. L. Unpublished Notes, Dept* of Electrical Engineering, 
University of Arizona, Tucson, Arizona, August 1970. 

7. Truxal, J. G. Control System Synthesis , New York: McGraw-Hill, 

1955. 

8. Eller, D. H., J. K. Aggarwol and H. T. Banks. "Optimal Control of 
Linear Time-Delay Systems," ’ IEEE Trans . on Automatic Control , 

Vol. AC-14, pp. 678-687, December 1969. 

9. Koivo, A. J. "Certainty Equivalence in Stochastic Linear Time Lag 
Systems," Proc . IEEE (letters), pp. 2061-2062, November 1969. 

10. Koepcke, R. W. "On the Control of Linear Systems with Pure Time 
Delay," Proc . JACC , pp. 397-403, 1964. 

11. Day, K. S. "Optimal Control of Linear Time Lag Systems," Ph.D. 
dissertation, Dept, of Elec. Eng'g., University of California, 

Davis, 1967. 

12. Kalman, R. E,. and R. W. Koepcke. "Optimal Synthesis of Linear 
Sampling Control Systems Using Generalized Performance Indexes," 
Trans . ASME , Vol. 80, p. 1820, 1958. 


139 



140 


* 

13. Joseph., F. D. and J. D, Tow* M On Linear Control Theory," Trans. 
A1EE, pt. XI, Vol. 80, p. 193, 1961. 

* 

14. Gunckel, T. L., IT, and G. F. Franklin, "A General Solution for 
Linear Sampled Data Control," J. Basic Eng^ ., Vol. 85, p. 197, 1963. 












■BIBLIOGRAPHY 


1. Ahmed, N, U. "Optimal Control of a Large Class of Stochastic Dif¬ 

ferential Systems with Finite Memory," Proc . 3rd Annual Aller- 
ton Conference on Circuit System and Theory , pp. 44-53. 

Urhana, Illinois, October 1969. 

2. Alekal, Y. "Feedback Controller Synthesis for Systems with Time 

Delay," Ph.D. Thesis, University of Minnesota, 1968. 

3. Alekal, Y. "Optimal Control of Linear Systems with Time Delay," 

Proc . National Electronics Conference , pp. 74-78, 1969. 

4. Alekal, Y. f P. Brunovsky and E. B. Lee. "Synthesis of Feedback 

Controllers for Time Delay Systems," Conference on Optimal 
Control and Minimal ‘ Surfaces , Tbilisi!!, U.S.S.R., September 
. 1969. 

5. Banas, J. F. and A. G. Vacroux. "Comments on ’Necessary Conditions 

for the Optimal Control of a System' with Time-Varying Transport 
Lags,’" IEEE Trans , on Automatic Control (Correspondence), 

Vol. AC-13, pp. 208-209, April 1968. 

6. Banas, J, F. and A. G. Vacroux, "On Linear Systems with a Time- 

Varying Delay and Isoperimetric Constraints," IEEE Trans , on 
Automatic Contro l (Correspondence), Vol. AC-13, pp. 439-440, 
August 1968. 

7. Banas, J. F. and A. G. Vacroux. "On the Optimal Aperiodic Control 

of Time Lag Systems," Second Asilomar Conference on Circuits 
and Systems , pp. 340-344, 1968, 

8. Banas, J, F. and A. G. Vacroux. "Discrete Optimal Control of Mul¬ 

tivariable Systems with Transport Lags," Proc . IFAC Symposium 
on Multivariable Control Systems , Duesseldorf, Germany, 1968. 

9. Banas, J. F. and A. G. Vacroux. "On the Optimal PW - PA Control 

of Time-Lag Systems," Proc . of 2nd Hawaii International Conf . 
on Systems Science , pp. 135-138, 1969. 

10. Banks, H. T. "Optimal Control Problems with Delays," Ph.D. Thesis 

Purdue University, 1967. 

11. Banks, H. T. "Necessary Conditions for Control Problems with Vari¬ 

able Lags," SIAM J. Control, Vol. 6, pp. 9-47, 1968. 

V 141 . . . - . : * V/- 



ft 


’ * 



142 


12. Becker, S, and P. V. Lopresti. "On Che Discrete Optimization of 

Linear Delay Systems," Proc . National Electronics Conference , 
pp. 125-130, 1967. 

13. Bellman, R. L. and K. L. Cooke. Differential-Difference Equations . 

New York: Academic Press, 1963. 

14. Bellman, R. L., H. H, Kagiwada and R. E. Kalaba. "Quasilineariza- 

tion and Estimation of Time Lags," Mathematical Biosciences l t 
pp. 39-44, 1967. 

15. Bendrikov, G. A. and F. B. Konev. "Investigation of High-Order 

System with, ’Pure' 'Delay Using the Method of Root Loci," 
Automation and Remote Control, pp. 502-511, April 1969. 

*- 

16. Bhatt, S. J. and C. S. Hsu. "Stability Criteria for Second-Order 

Dynamic Systems with. Time-Lag," Journal of Applied Mechanics , 
Vol. 33, Series E, No. 1, March. 1966. 

17. Boyanovitch, D. "Variational Problems with Argument Lags," 

Research Notes RN-179 , Grumman Aircraft Engineering Corpora¬ 
tion, Research Dept., March 1964. 

# i 

18. Brookes, B. E. and R. A. Johnson. "Limit Cycles in Time-Delay 

Relay Systems," Int’l. J. on Control, Vol. 9, No. 4, pp. 387- 
398, 1969. v 

i 

19. Brown, R. G. and G. L* Hartmann. "Kalman Filter with Delayed 

States as Observables," Proc . National Electronics Confer¬ 
ence , pp. 67-72, 1968. 

20. Buckalo, A. F. "Controllability of Systems with Time Lag," 

Sc. M. Thesis, Brown University, Providence, R. I., 1967. 

21. . Buckalo, A. F. "Explicit Conditions for Controllability of Linear. 

Systems with Time Lag," IEEE Trans . on Automatic Control 
(Short Papers), Vol. AC-13, pp. 193-195, April 1968. 

22. Buckley, P. S. "Automatic Control of Processes with Dead Time.," 

Proc . 1st IFAC Congress (Moscow), Vol. 1, p. 33, 1960. 

23. Budelis, J. J. and A. E. Bryson, Jr., "Some Optimal Control Re¬ 

sults for Di*fferential-Diff erence Systems," IEEE Trans . on 
Automatic Control (Short Papers), Vol. AC-15, pp. 237-241, 
April 1970. 

24. Buyakas, V. I, and A. A. Piontkovskii. "Analytical Considerations 

in the Design of Regulators for Systems with Time-Delay in a 
Servomechanism," Automation and Remote Control , pp. 163-165, 
February 1969. 




I 


143 

25. Choksy, N. E. ’’Analytic Determination of the Stability of Servo¬ 

mechanisms and Other Automatic Control Systems Having Trans¬ 
cendental Characteristic Equations,” Ph.D. dissertation, 
University of Wisconsin, 1955. 

* 

26. Chyung, D. U. "Discrete Systems with Time Delay," .'Proc. 5th 

Annual Allerton Conference on Circui t System and Theory, 

PP* 222-227, Urbana, Illinois, October 1967.’ 

* 

27. Chyung, D. H. "Linear Time-Lag Systems with Side Constraints," 

IEEE Trans , on Automatic Control CShort Papers), Vol. AC-12, 
pp. 434-435, August 1967. 

* a 

28. Chyung, D. H. "Linear Time-Lag Systems with Side Constraints," 

Proc . JACC . pp. 470-473, 1967. 

, ♦ * 

29. Chyung, D. H. ’’Discrete Optimal Systems with Time Delay," IEEE 

Trans , on Automatic Control (Correspondence), Vo,V AC-13, 
p. 117, February 1968. 

30. Chyung, D» E. "Optimal Time-Lag Systems with Sensitivity Con¬ 

straint," IEEE Trans , on Automatic Control (Correspondence), 

Vol. AC-13, p. 736, December 1968. 

31. Chyung, D. H. "Discrete Systems with Delay- in Control," IEEE 

Trans , on Automatic Control (Correspondence), Vol. AC-14, 
pp. 196-197, April 1969. 

32. Chyung, D. H. "On the Controllability of Linear Systems with 

Delay- in Control," Proc , 7th Annual Allerton* C onference on 
Circuit System and Theory , pp. 19-22, October 1969. 

33. Chyung, D. H. "On the Controllability of Linear Systems with Delay 

in Control," IEEE Tran s, on Automatic Control (Correspondence). 
Vol. AC-15, pp. 255-257,-April 1970. " 

34. Chyung, D. H. "A Pursuit-Evasion Game of Linear Systems with Delay," 

IEEE Trans , on Automatic Control (Correspondence), Vol. AC-15, 
pp. 387-389, June 1970. * 

35. Chyung, D. H. and E. B. Lee. "Linear Optimal Systems with Time 

Delay," SIAM J. Control , Vol. 4, pp. 548-575, November 1966. 

36. Chyung, D. H. and E. B. Lee. "Optimal Systems with Time Delay," 

Proc. 3rd IFAC Congress (London), 1966. 

f 

37. Chyung, D. H. and E. B* Lee. "Control of Linear Time Delay Systems 

with Essentially Linear Cost Functional.s," Differential 
Equations and Dynamical Systems , J. K. Hale and J. P. LaSalle, 
eds. New York: Academic Press, 1967. 





38 # , Chyung, D. H, and E. B. Lee. "Delayed Action Control Problems," 
Proc . 4 th. 1FAC Congress (Whrsaw), 1969. 

39. Chyung, D. II, and E. B. Lee. "Delayed Action Control Problems," 

Automation , Vol. 6, No. 3, pp. 395-400, May 1970. 

* 

40. Ciletti, M. D. "A Differential Game with an Information Time Lag, 

Proc . of the First Int*1 . Conferenc e on the Theory and Appli¬ 
cation of Differential Games , University of Massachusetts, 
Amherst, Massachusetts, September 1969. 

41. Ciletti, M. D„ "A Differential Game with an Information Lag," 

Proc . 3rd H awaii Int f l . Conf . on System Sciences , pp. 882-885, 
1970. 

42. Cohen, G. M. "Shift Matrices and Their Application," IEEE Trans . 

on Automatic Control (Short Papers), Vol. AC-11, pp, 543-55Q, 

. July* 1966, 

» 

43. Das, p. C. and R. R. Sharma. "On Optimal Controls for Measure 

Delay-Differential Equations," SIAM J. Control , pp. 43-61, 
February* 1971. 

44. Day-, K. S. "Optimal Control of Linear Time-Lag Systems," Ph.D. 

dissertation, Dept, of Elec. Eng’g., University of California, 
Davi,s,, 1967. 

45. Day, K. S. and T. C, Hsia. "Optimal Control of Linear Time Lag 

Systems," Proc . JACC , pp. 1046-1055, 1968. 

• * * 

• i » 

46. ' Degtyarev, G. L. and T. K. Sirazetdinov. "Optimal Control of One- 

Dimensional Processes with Retarded Arguments," Automation 
and Remote Control , pp. 28*-36, January 1970. 

47. Diver, R. D. "Existence and Stability of Solutions of a Delay 

Differential System," Archives for Rational Mechanics and 
Analysis , Vol. 10, No, 5, pp.401-426, 1962. 

48. Eisenberg, L. "Stability of Linear Systems with Transport Lag," 

XEEE Trans , on Automatic Control , Vol. AC-11, pp. 247-254, 
April 1966. 

49. Eisenberg, L. "Stability of Linear Systems with Transport Lag," 

IEEE Int 1 ! *- Convention Record , Pt. 6, Vol. 14, pp. 207-218, 
1966. 

50. Eisenberg, L. and J. J. Padalino. "Root-Locus Method for Systems 

with Transport Lag," Proc .* National Electronics Conference , 
pp. 661-666, 1966. 







% * ' 

'♦ . * * f 


. * • 145 

* 

51. Eller, D, H* ’’Optimal Control of Time Delay Systems,” Ph.D. dis¬ 

sertation, Dept, of Elec. Eng’g., University of Texas, Austin, 
Texas, 1969. 

52. Eller,* D. H., J. K. Aggarwol and H. T. Banks, ’’Optimal Control of 

Linear Time-Delay Systems,” IEEE Trans , on Automatic Control , 
Vol. AC-14, pp. 678-687, December 1969. 

53. Falb, P. L. and D, L. Kleinman. ’’Remarks on the Infinite Dimen- 

. sional Riccati Equation,” IEEE Trans, on Automatic Control, 

Vol. AC-11, pp. 534-536, 1966. 

54. Faure, F. and F. J. Evans. ’’Identification of Process Delay Time,” 

IEEE Trans , on Automatic Control (Correspondence), Vol. AC-14, 
pp. 421-422, August 1969. 

55. Ferrell, M. R. "Remote Manipulation with Transmission Delay,” 

IEEE Transactions on Human Factors in Electronics , Vol. HFE-6, 
No. 1, pp. 24-32, September 1965. 

56. Fountain, F. H. and C. A. Halijak. "An Approximate Identity Oper¬ 

ator for Continuous Servo mechanism with. Time Lag,” Proc , 4th 
Annual Allerton Conf . on Circuit System and Theory , p. 914, 
Urbana, Illinois, October 1966. 

.57. Fuller, A. T. "Optimal Nonlinear Control of Systems with Pure 

Delay,” Int’1 . J_. Control , Vol. 8, No. ?, pp. 145-168, 1968. 

58. Friedman, A. "Optimal Control for Hereditary Processes,” Arch . 

Rational Mech . Anal ., Vol. 15, pp. 396-414, 1964. 

59. Gabasov, R. and S. V. Churakova. "Necessary Optimality Conditions 

in Time-Lag Systems," Automatio n and Remote Control , pp. 37- 
54, January 1968. 

60. Gabasov, R. and S. V. Churakova. "Sufficient Conditions for 

Optimality in Systems with a Delay,” Automation and Remote 
Control , pp. 193-209, February 1968. 

61. Giloi, W. "Optimized Feedback Control of Dead Time Plants by Com¬ 

plementary Feedback,” Proc . JACC , pp. 211-217, 1963. 

62. Gruber, M, "Path Integrals and Lyapunov Functionals," IEEE 

Trans, on Automatic Control, Vol. AC-14, pp. 465-475, October 
1969. 

63. Guardabassi, G. and S. Rinaldi. "Optimization of Discrete Systems 

With Delayed Control Variables —• A Structural Approach," 
Automation and Remote Control , pp< 1063-1067, July 1968. 







146 

64. Halanay*, A. "Periodic Solutions of Linear Systems with Lag," 

Revue de Nath , Pure et Appliques , Acad. R. S. R., Vol*. VI, 

No. 1, 1961. 

65. Halanay, A. Differential Equations --■> Stability , Oscillations , 

Time- Lags . New York: Academic Press, 1966. 

66. Halanay, A. "Differential Games with. Delay," SIAM J. Control , 

pp. 579-593, November 1968. 

k 

67. Halanay, A. "Optimal Controls for Systems with Time Lag," SIAM 

’ Control , pp. 215-234, May* 1968. (See also "erratum," 

SIAM J. Control , pi 142, February 1971.) 

68. Hale, J. K. "Averaging Methods for Differential Equations with 

Retarded Arguments and a Small parameter," Tech . Report No . 
64-1 , Center for Dynamical Systems, Division of Applied Mathe- ’ 
mati.cs, Brown University, Providence, R. I., November ,1964. 

69. Hale, J. K. "Sufficient Conditions for Stability and Instability 

of Autonomous Functional-Differential Equations," .J. Differ¬ 
ential Equations, 1, pp. 452-485, 1965. 

70. Hamza, M. H. "Extremum Control in the Presence of Variable Pure 

Delay," Theory of Self-Adaptive Control Systems , pp. 290-298, 
New York: Plenum Press, 1966. 

71. Hiratsuka, S. and A. Ichikawa. "Optimal Control of Systems with 

Transportation Lags," IEEE Trans , on Automatic Control , 

Vol. AC-14, pp. 237-247, June 1969. 

72. Horwitz, L. B. "On Computing Optimal Control Signals for Systems 

with. Time Delays," Proc . of 3rd Hawaii Int t l . Conf . on 
System Sciences , pp. 363-366, 1970. 

73Jen-Wei, Chang. "The Problem of Synthesizing an Optimal Controller 
in Systems with Time Delay," Automation and Remote Control , 
pp. 121-125, February 1962.' 

74. Johnson, R. A. "Comment on the Approximation of a System with 

Delay by an Ordinary Differential System," IEEE Trans , on 
Automatic Control (Correspondence), Vol. AC-13, pp. 755-756, 
December 19;6S. 

75. Kalachev, M. G. and A. M. Petrovskii. "Effect of Noise on an 

Ideal Relay which Includes Delayed Feedback," Automation and 
Remote Control , pp. 1894-1900, November 1966. 

76. Kashiwagi, Y. and I. Flugge-Lotz. "Stability of Linear Systems 

with Time Delay," TR-156 , Stanford University, 1965. 





77. Kashiwagi, Y. and J. D. Shaughnessy. ’’Stability of Linear Systems 

with. Multiple Delays," Proc . of the Canadian Congress of 
Applied Mechanics , Quebec, Canada, May 1967. 

78. Kashiwagi, Y’. and J. D. Shaughnessy. "Stability Indicative Func¬ 

tion and its Application to the Linear Systems with Multiple 
Delays-," Proc . of Hawaii Int’ 1 . Conf . on System Sciences , 
pp. 189-192, 1968. 

79s Khalanl, A. "Absolute Stability .of Certain Non-Linear Control 

Systems with Lags," Automation and Remote Control , pp. 268- 
279, March 1964. 

80. Khalani, A. "Absolute Stability of Non-Linear Control Systems 

with Delay Arguments," Automation and Remote Control , 
pp. 1251-1256, August 1964. 

81. Kharati'shvilli, G. L. "The Maximum Principle in the Theory of 

Optimal Processes with Time Lags," Dokl . Akad . Nauk SSSR , 

Vol. 136, No. 1, pp. 39-42, 1961. 

•82. Kharatishvilli, G. L. "A Maximum Principle in External Problems 
with. Delays," Proc . Conf . on Mathematical Theory of Control , 
University of Southern Calif., A. V. Balakrishnan and L. W. 
Neustadt, eds., New York: Academic Press, pp. 26-34, 1967. 

83. Khatri, H. C. "Optimum Control of Linear Systems with Transpor¬ 

tation Lag," £. of Basic Eng'g ., pp. 385-392, June 1967. 

84. Kira, M. and C. H. Yeh. "Stability Analysis of Time Lag Systems," 

Proc . IEEE (letters), p. 1366, November 1964. 

* 

85. Kirillova, F. M. and S. V. Curakova. "On the. Problem of Controlla¬ 

bility of Linear Systems with. After-Effect," (Russian), Diff . 
Urav ., Vol. 3, pp. 436-445, 1967. 

86. Kitamura, S., K. Hirai and M. Nishimura. "Stability of a Control 

System with Several Nonlinear Elements and Time Lags," Tech . 
Report 758 , Osaka Univ., - Vol. 17, pp. 93-102, 1966. 

87. Kleinman, D. L. "Optimal Control of Linear Systems with Time 

Delay and Observation Noise," IEEE Trans . on Automatic Control 
(Short Papers), Vol. AC-14, pp. 524-527, October 1969. 

88. Koepcke, R. W. "On the Control of Linear Systems with Pure Time 

Delay," Proc . JACC , pp. 397-403, 1964. 

'•■V . •' . . . <%■ 

8 % Koepcke, R. W. "On the Control of Linear Systems with Pure Time 
Delay," J. of Basic Eng’g., pp. 74-78, March 1965. 







148 


90. Koivuniemi, A. J. and P. F. Schweisier. M 0n Optimal Control, Sys¬ 
tems, with. Adjustable Delay Times,” * Proc . National Electronics 
Conference , pp. 120-124, 1967. 

•91. Koivuniemi., A. J. and P. F. Schweiser. "Design of Optimal Control 
Systems with Pure Time Delay," Proc . of Hawaii Int*l . Conf , 
on System Sciences , pp. 33-36, 1968. 

92. Koivuniemi, A. J. and R. L. Stoller. "On Estimation of State 

Variables in Time-Lag Systems," 2nd Annual Princeton Confer- 
ence on Information Sciences and Systems, Princeton, N. J., 

1968. . 1 • 

.93. Koivo. A. J. "On Optimum Control of Systems with Time Lag and 

State Variable Constraints," Proc . National Electronics Con¬ 
ference , pp, 130-135, 1968. 

94. Koivo, A. J. "Certainty Equivalence in Stochastic Linear Time 

Lag Systems," Proc. IEEE (letters), pp. 2061-2062, November 

1969. 

* % 

95. Koivo, A. J. "On Identification of Time Lag in Dynamical Systems," 

Proc . JA'CC , pp. 782-786, 1969. 

r 

96. Koivo, A. J. and R. L. Stoller. "On Least Squares Estimation in 

Non-Linear Dynamical Systems with. Time Delay," Proc. JACC, 

pp. 116-121, 1968. • ~ 

97. Koralev, N. A. "Compensation of Delay in a Relay System," Automa¬ 

tion and Remote Control , pp. 523-530, May 1961. 

98. Krall, A. M. "Stability Criteria for Feedback Systems with Time 

Lag," SIAM J. Control , Vol. 2, No. 2, pp. 160-170, 1964. 

99. * Krall, A. M. "The Asymptotes of the Time Lag Root Locus," SIAM 

J. Control , Vol. 2, No. 3, pp. 370-372, 1964. 

100. Kramer, J. D. "On Control "of Linear Systems with Time Lags," 

Information and Control , Vol. 3, pp. 299-326, 1960. 

101. Krasovskii, N. N. "Analytic Construction of an Optimal Regulator 

in a Systenrwith Time Lags," Principles of Math, and Mech,, 
Vol. 26, No'. 1, pp. 50-67, 1962. 

102. Krasovskii, N. N. "Optimal Control Under Conditions of Lagging 

Feedback," Automation and Remote Control , pp. 931-945, 

August 1963. 

103. Krasovskii, N. N. "Optimal Processes in Systems with Time Lag," 

Proc . 2nd IFAC Congress, Vol. I, 1963. 




149 


104. Krasovskii, N. N. "The Approximation of a Problem of Analytic 

Design of Controls in a System with Time Lag," Principles of 
Math. , and Mach ., Vol. 28, No. 4, pp.' 716-724, 1964. 

105. Krasovfckii*, N. N. "Optimal Processes in Systems with Time Lags," 

Proc . 2nd 1FAC Congress, Basel, Switzerland, 1963. 

106. Krasovskii, N. N. and Y. S. Osipov. "The Stabilization of Motion 

of a Controllable Object with Time Lag in Control System," 

* Tekhnicheskaya Kibernetika , No. 6, 1963. 

107. Kurzweil, F., Jr. "The Control of Multivariable Processes in the 

Presence of Pure Transport Delays," IEEE Trans , on Automatic 
Control , Vol. AC-8, p. 27, January 1963. 

108. Kwakernaak, H. "Optimal Filtering in Linear Systems with Time 

Delays," IEEE Trans , on Automatic Control , Vol. AC-12, 
pp. 169-173, April 1967. 

4 - 

109. Larson, R. E. and C. H. Wells. "Combined Optimum Control and 

Estimation of Serial Systems with Time Delay," Proc. JACC, 
pp. 23-33, 1969. 

110. Lawrence, G. A. "Optimal Control of.Systems with Time Delay," 

Ph.D. dissertation, Northwestern University, Evanston, Illinois 
June 1969. 

111. Lee, E. B. "Variational Problems for Systems Having Delay in the 

Control Action," IEEE Trans. on Automatic Control (Short 
Papers), Vol. AC-13, pp. 697-699, December 1968. 

112. Lee, E. B. "Optimal Control of Systems with Time Delays," Proc . 

JACC , pp. 451-452, 1969. 

. v 

113. -Lee, G. M. "Non-Linear Filtering with Observational Delays," 

Proc. 3rd Hawaii Int'l. Conf. on System Sciences, pp. 469-472, 
1970. • 

114. Leonov, N. N. "Difference-type Relay External Systems with Trans¬ 

port Delays," Automation and Remote Control , pp. 1780-1789, 
November 1969, 

115. Loo, S. G. "Stability of Linear Stationary Systems with Time 

Delay," IntM . J. Control , Vol. 9, No. 1, pp. 387-398, 1969. 

116. Lopresti, P. V. and S. L. Uppal. "Closed Loop Optimization of 
Linear Delay Systems," Proc. National Electronics Conference, 
p. 824, 1967. 








150 


117. Lorchi.rachoonkul, V. "Deadbeat Response of Time-Delayed Systems 

V.ith. Minimal Overshoot Compromise," IEEE Trans , on Automatic 
Control (.Correspondence), Vol. AC-12, pp. 468—469, August 1967. 

118. MacKinnon, D. "Optimal Control of Systems with Pure Time Delays 

Using a Variational Programming Approach>" IEEE Trans . on 
Automatic Control , Vol. AC-12, pp. 255-262, December 1967. 

119. Markushin, E. M. and S. N. Shimanov. "The Approximate Analytic' 

Design of a Control for an Equation with Time Lag," 

Differentslal'nye Uravneniya , 1, No. 5, 1965. 

120. Markushin, E. M. and S. N. Shimanov. "Approximate Analytic Design 

of Controls for Systems with Time Lag," Automation and Remote 
Control , pp. 367-374, March 1968. 

121. McAulay, R. J. ."A Gradient Method for Systems with Time Delays 

and its Application to Waveform Design," IEEE Trans , on 
Automatic Control , Vol. AC-14, pp. 230-236, June 1969. 

122. Meerov, M. V. "Some Peculiarities of the Structure of Multiple 

Control Systems," Proc. 1st IFAC Congress (Moscow), pp. 85- 
89, 1960. ' ™ 

123. Milyutkin, V< P. "Maximum Principle for Problems with Delays, 

Fixed Durations, and Free Trajectory Right Ends," Automation 
and Remote Control , pp. 882-889, June 1968. 

124. Muellerj T. E. "Optimal Control of Linear Systems with Time Lag," 

Coordinated Science Lab . Report R-254 , University of Illinois, 
Urbana, Illinois, June 1965. 

125. Mueller, T. E. "Optimal Open-Loop Control of Linear Systems with 

Time Lag," Proc . 3rd Annual Allerton C onf . on Circuit System> 
and Theory , pp. 339-345, October 1965. 

126. Murray, T. "Bang-Bang Rate Damping with Time Delay and Time Lag," , 

IEEE Trans , on Automatic Control (Correspondence), Vol. AC-11, 
p. 137, January 1966. ' - 

127. Mufti, I. H. "The Stability of Systems with Lag," Mech . Eng . 

Report MK-16, Nat. Research Council of Canada, Ottawa, March 
1965. 

128. Mufti, I. H. "A Note on the Stability of an Equation of Third 

Order with Time Lag," IEEE Trans . on Automatic Control 
Correspondence), Vol. AC-9, pp. 109-191, April 1964. 

129. Norosel'tsev^, V.. N. "Optimal Control in the Presence of Delay," 

Automation and Remote Control, pp. 1395-1397, November 1964. 


I 


151 


130. Oetker, R. "On the Control of Sectors with Dead Time," Proc . 1st 

IFAC Congress (Moscow), pp. 22-25, ‘I960. 

* 

131. Osipov, Y. S. "The Stabilization of Controlled Systems with Time 

Lag," Differentgial'nye Uravneniya , 1, No. 5, 1965. 

# 

132. Oguztoreli, M. N. "A Time Optimal Control Problem for Systems 

Described by Differential-Difference Equations," SIAM J. 
Control , VoT. 1, pp. 290-310, August 1963. 

%■ 

133. Oguztoreli, M. N. "Relay Type Control with Retardation and Switch¬ 

ing Delay," SIAM J. Control , Vol. 1, No. 3, pp. 275-289, 1963. 

134. Oguztoreli, W, N. "Optimal Pursuit Strategy Processes with Re¬ 

tarded Control Systems," SIAM J. Control , Vol. 2, No. 1, 
pp. 89-105, 1964. 

* 

135. Oguztoreli, M. N. Time Lag Control Systems . New York: Academic 

Press., 1966. 

136. Ozi'ganova, I. A. "On the Theory of Optimal Control of Systems with 

Time Lag," Seminal' on Differential Equations with Deviating 
Arguments , University of the Friendship of Peoples, Moscow, 
Vol. IT, pp. 136-145, 1963. See also "On the Theory of Optimal 
Control for Problems with Time Lag," Thesis, University of the 
Friendship of Peoples, Moscow, 1966. 

137. Popov, V. M. and A. Khalani. "On the Stability of Non-Linear 

Automatic Control Systems with Lagging Argument," Automation 
and Remote Control, pp. 783-786, July 1962. 

138. Popov, V. M. and A. Khalani. "A Problem in the Theory of Time- 

Delay Optimum Systems," Automation and Remote Control , 
pp. 129-131, February 1963. 

139. Priemer, R. and A. G. Vacroux. "Estimation in Linear Discrete 

Systems with Time Delay," Proc . National Electronics Confer ¬ 
ence , pp. 64-66, 1968. 

140. Priemer, R. and A. G. Vacroux. "Smoothing in Linear Discrete Sys¬ 

tems with Multiple Time Delays," Second Asilomar Conference 
on Circuits and Systems , pp. 136-140, 1968. 


141. Priemer, R. and A. G. Vacroux. "Estimation in Linear Discrete 

sy stems with Multiple Time Delays," IEEE Trans , on Automatic 
Control (Short Papers), Vol. AC-15, pp. 237-241, April 1970. 

142. Priemer, R. and A. G. Vacroux. "Fixed Lag Smoothing in Linear 

Discrete Systems with Time Delays," Proc . 7th Annual Aller- 
ton Conf. on Circuit System and Theory , pp. 389-390,-Urbana, 
Illinois, October 1969. . 




152 


143. pri.emer, R. and A. G. Vacroux. "On Smoothing in Linear Discrete 

Systems with Timd Delays," Xnt 1 ! . .J. Control , Vol. 13,-No. 2, 
pp. 299-303, February 1971. 

144. Pyatnitskii, F;. S. "On the Structural Stability of Single-Loop 

Control Systems, in the Presence of Lag," Automation and Re¬ 
mote Control , pp. 787-797, July 1962. 

145. Pyatni.tski.i, E. S. "The Upper Bound on the Degree of Stability 

for Time-Lag Systems," Automation and Remote Control, pp. 434- 
439, April 1963. 

146. Ragg, B. C. "Necessary Conditions for the Optimal Control of a 

. System with. Time-Varying Transport Lags," IEEE Trans , on 
Automatic Control (Short Papers), Vol. AC-11, pp. 738-740, 

July 1966. See also Correspondence IEEE Trans , on Automatic 
Control , Vol. AC-13, pp. 208-209, April 1968. 

147. Ray, K. H. and M. A. Solimar. "The Optimal Control of Processes 

Containing Pure Time Delays," Proc . JACC ,.pp. 476-484, 1970. 

148. Razumikhin, B. S. "Stability of Nonlinear Automatic Control 

Systems--with Lag," Inzhenernyl sb ., Vol, 29, 1960. 

149. Reeve, P. J. "A Method of Approximating to Pure Time Delay," 

Int* 1 . J. Control , Vol. 8, No. 1, pp. 53-63, 1968. 

150. Reeve, P. J. "Time Optimal Control of Linear Systems with Delay,"_ 

Int 1 ! . J^. Control , Vol. 11, No. 4, pp. 659-682, April 1970. 

See also Correspondence Int’!. J. Control , Vol. 13, No. 1, 
pp. 201-203, January 1971. 

151. Rekasius, Z. V. and G. A. Lawrence. "Minimum Energy Control of 

Systems with Time Delay," IEEE Trans , on Automatic Control 
(Short Papers), Vol. AC-15, pp. 365-368, June 1970, 

152. Repin, I. M. "On the Approximate Replacement of Systems with 

Lag by an Ordinary Dynamical System," Principles of Math . 
and Mech ., Vol. 29, No. 2, pp. 254-264, 1965. 

153. Reowick, J. B. "A Delay Regulator," Proc . 1st IFAC Congress 

(Moscow), 1960. 

154. Roberts, A. P. "An Adaptive Lagging System," Theory of Self- 

Adaptive Control Systems , pp. 198-203, New York: Plenum 
Press, 1966. 

155. Robinson, W. R. "Identification and Parameter Optimization of 

Linear Systems with Time Delay," M. A. Sc. Thesis, Dept, 
of Elec. Eng'g., University of British. Columbia, Vancouver, 

B. C., Canada, June 1966. 


£ * - 


153 

* 

156. Robinson, W. R. and A. C. Soudack. "A Method for the Identifica¬ 

tion of Time Delays l,n Linear Systems,' 1 IEEE Trans , on Auto¬ 
matic Control (Short Papers), Vol. AC-15, pp. 237-241, April 
1970. , 

» * 

* 

* t 

157. Ross, P, W. "Optimal Control of Systems Described by Differential- 

Difference Equations," Ph.D. dissertation, Dept, of Elec. 
Eng'g., Stanford University, Stanford, Calif., 1968, 

** 

158. Ross, D. VT. "Optimal Control of.Systems with Transport Lag with 

Applications to Process Control," Proc . of 2nd Hawaii Int'l . 
Conf . on Systems Science , pp. 131—134, 1969. 

159. Ross, D. W. and I;. Iflugge-Lotz. "An Optimal Control Problem for 

Systems with Differential-Difference'Equation Dynamics," 

SIAM J, Control , pp. 609-623, November 1969. 

* * 

160. Rozenvasser, E. N. "The Theory of Linear Systems with Stationary 

Delay and a Periodically Varying Parameter," Automation and 
Remote Control , pp. 956-964, July 1964. 

* 

161. Salukvadze, M. E. "Concerning the Synthesis of an Optimal Control¬ 

ler in Linear Delay Systems Subjected to Constantly Acting 
Perturbations," Automation and Remote Control , pp. 1495-1501, 
December 1962. 

162. Sannuti, P. "Near Optimum Design of Time Lag Systems by Singular 

Perturbation Method," Proc . JACC ,.pp. 489-496, 1970. 

163. Schweizer, P. F. "Design of Optimal Control Systems with Time 

Delay," Ph.D. dissertation, Dept, of Elec. Eng'g., Purdue 
University, August 1968. 

164. Schweizer, P. F. and A. J. Koivo. "On Optimal Systems with Vary¬ 

ing Time Lag and Stability Constraints," Proc . National 
Electronics Conference , pp. 68-73, 1969. 

165. Sebesta, H. R. "Analysis and Design of Optimal Control Systems 

for Dynamical Processes 'with Time Delays," Ph.D. disserta¬ 
tion, University' of Texas, 1966, 

166. Sebesta, H. R. and L. G. Clark. "On Optimization and. the Guidance 

Problem for Systems with Transport Lags," Southeastern Sym¬ 
posium in Missile and Aerospace Sciences , American Astro- 
nautical Society, Huntsville, Alabama, December 1966. 

167. Sebesta, H. R. and L. G. Clark. "On the Optimal Control Problem 

for Dynamical Processes with Variable Delays," Proc . JACC , 
pp. 326-334, 1967. 




154 


168. Sebes.ta, FL R. and L. G, Clark, "On the Optimal Control Problem 

for Dynamical Processes with Variable Delays," £. of Basic 
Eng 1 g ., pp. 181-186, June 1968. 

169. Seborg, D. E. "Stability of Nonlinear Systems Containing Time 

Delays," Ph.D, dissertation, Dept, of Chem. Eng'g., Princeton 
University, Princeton, N. J., June 1969. 

170. Seborg, D. E. and E. F. Johnson. "Construction of Regions of Sta¬ 

bility for Nonlinear Systems Containing Time Delays," Proc . 
JACC, pp. 246-253, 1970. 

171. Shak, F. FC. "Optimal Control of Discrete Systems with Time Lag," 

, Automation and Remote Control , pp. 1065-1073, July 1970. 

172. Shigin, E. K. "Automatic Control of a Pure-Lag Plant by a Control¬ 

ler with Switchable Parameters, I," Automation and Remote 
Control , pp. 1612-1618, October 1965. 

173. Shigin, E. K. "Automatic Control of a Plant with a Pure Time Lag 

by a Regulator with Switchable Parameters, II," Automation 
and Remote Control , pp. 999-1007, June 1966. 

174. Shiraanov, S. N. "Instability of Motion of Systems with Time Lag," 

Applied Math , and Mech ., Vol. 24, No. 1, pp. 70-81, 1960, 

175. Shimanov, S. N. "The Theory of Linear Differential Equations with 

periodic Coefficients and Time Lag," Applied Math, and Mech., 
Vol'. 27, No. 3, 1963. 

176. Shimeraura, E„, Y. Yoshida, K. Kawamura and Y. Komatsu. "Optimal 

Control of Linear Systems with Time Delay," £. Soc . Instru¬ 
ment and Control Engineers (Japan), Vol. 3, pp. 415-421, June 
1964. 

177. Shimemura, E. and Y. Komatsu^ "Optimal Problems in Systems with 

Retarded Control," Bulletin Sci . Eng*g . Res . Lab ., Waseda 
University, No. 28, pp. 23-25, September 1964. 

178. Simpson, R. S. and Kl C. Houts. "A Definition of Average Time 

Delay for a Linear System," Proc . of IEEE (letters), pp. 1733- 
1734, October 1967. 

179. Soliman, J. I. and Fh Kwoh. "A Frequence Stability Criterion for 

a Class of Non-Linear Sample-Data System •with Pure Delay," 

Int'1 . J. Control, Vol. 7, No. 6, pp. 579-589, 1968, 

180. Strakhov, V. P. "Analysis of Digital Time-Lag Servo Systems," 

Automation and Remote Control , pp. 1119-1125, August 1964. 






155 


181. Tahim, G, S. ’’Optimum Control of Linear Systems with Time Lag," 

Coordinated Science Lab ., Report Rt 242 , University of Illinois, 
Urbana, Illinois, January 1965. 

182. Tahim; G. S. "Controllability of linear Systems with Time Lag," 

Proc . 3rd Annual Allerton Conf . on Circuit System and Theory , 
pp. 346-352, Urbana, Illinois, October 1965. 

183. Tahim, G. S. "The Problem of Lure in Systems with Time Lag," 

Proc . 5th, Annual Allerton Conf . on Circuit System and Theory , 
pp* 259-262, Urbana, Illinois, October 1967. 

* » 

184. Tahim, G. S. ”0n the Approximation of a System with Delay by an 

Ordinary Differential System," IEEE Trans , on Automatic Con¬ 
trol (Correspondence), Vol. AC-12, pp. 794-795, December 1967. 

185. Tai,t, K. E. and R, B. Stanton. "The Inter-Relation of Sampling 

and Transport Lag in Determining Control System Stability," 

Int f l. J. Control , Vol. 5, No. 2, pp. 97-122, 1967. 

186. Tal, J, "Low Sensitivity* Design of Systems with. Time Lag," M. S. 

Thesi.s, University of Minnesota, 1969. 

M. 

187. Tal, J. and S'. J. Kahne. "Design of Time Lag Systems with Sensi¬ 

tivity Constraints," Proc . JACC , pp. 459-462, 1969. 

i 

188. Teverovskii, V. I*. "Periodic Operating Mode of Relay Systems with 

Variable ’Pure’ Lags," Automation and Remote Control , pp, 73*= 
79, January 1966. 

189. Thompson, VT. M. "Force-Reflecting S'ervomechanlsms with Signal 

Transmission Time Delay," Ph.D. thesis, Illinois Institute 
of Technology, 1967. 

190. * Thompson, ty. M., A. G. Vacroux and C. H. Hoffman. "Application of 

Pontryagin’s Time Lag Stability Criterion to Force-Reflecting 
Servomechanisms," Proc . JACC , pp. 432-443, 1968. 

191. TomlinvSon, G. K. and M. J. Sommerville. "Low Pass Approximations 

to a Time Delay Characteristic," Journal of Electronics and 
Control , Vol. 15, No. 5, pp. 495-512, 1963. 

192. Vasil’eva, N. P. and I. Gashkovits, "Conditions for Exact Reali¬ 

zation of Sequential Functions by Various Sets of Actual 
Logic Elements, Account Being Taken of Lags," Automation and 
Remote Control , pp. 1535-1545, f September 1965. 

193. Venkatesh, Y. V. and B. L. Deekshatulu. "Analysis of Time Delay 

Systems," Int’l. J. Control, Vol. 4, No. 4, pp. 337-356, 

1966. 




156 


/ , 

* 

194*. Voland, J, "On the Linear Slope Delay Approximation," ?roc . IEEE 
(letters), pp. 2059-2060, Novembar 1967. 

195. Voland, J. "Addendum to ’On the Linear Slope Delay Approximation,*" 

Proc. IEEE (letters), pp. 748-750, April 1968. 

$ 

» 

196. Voland, J. "A Further Note on the Linear Slope Delay Approxima- 

tion," Proc . IEEE (letters), pp. 1356-1357, August 1968. 

& 

197. Walker, J. A. "Stability of Feedback Systems Involving Time Delays 

and a Time Varying Nonlinearity*," Int’l. J. Control. Vol, 6, 

No. 4, pp. 365-372, 1967. 

,198. Wang, P. K. C. "Asymptotic Stability of a Time-Delayed Diffusion 
System," .J. Appl . Mech . (ASME), Vol. 30, Series E, No. 4, 
pp. 500-504, 1963. 

» 

199. Wang, p. K. C. "Optimum Control of Distributed Parameter Systems 

* with. Time Delays," IEEE Trans , on Automatic Control . Vol, AC-9, 
pp. 13-22, January 1964. 

200. Wang, P.' K. C. "On the Almost Sure Stability of Linear Time-Lag 

Systems with Stochastic Parameters," Int’l . J. Control . Vol. 2, 
No. 5, pp. 433-440, 1965. 

* * 

201. Wang, P. K. C. and M. L. Bandy. "Stability of Distributed Para- 

• meter Processes with Time Delays," Journal of Electronics 
* and Control , Vol. 15, No. 4, pp. 343—362, 1963. 

202. Wells*, W*'R. and Y. Kashiwagi. "A Time Optimal Control Study of a 

Second Order Linear System with Delay," Proc. JACC, pp. 644— 
649, 1968. 

.203• Weiss, L. "On the Controllability of Delay-Differential Systems," 
Princeton Conference on Information Sciences and Systems , 
Princeton, N. J., March 1967, 

• 

204. Weiss, L. "On the Controllability of Delay-Differential Systems," 

SIAM J. Control , pp, 575-587, November 1967. 

205. Weiss, L. "New Aspects and.. Results in the Theory of Controllabil¬ 

ity," Proc . JACC . pp. 453-456, 1969. 

206. Weiss, L. "Lectures on Controllability and Observability," 

£• JL» H- I* Seminar on Controllability and Observability , 
pp. 205-289, Rome: Edizioni Cremonese, 1969. 

207. Weiss, L. "An Algebraic Criterion for Controllability of Linear 

Systems with Time Delay," IEEE Trans , on Automatic Control 
(Short Papers), Vol. AC-15, pp. 443-444, August 1970. 









* 

* * 


157 

208. Wei,ssi. R. "Transportation Lag~~An Annotated Bibliography," IRE 

Trans . on Automatic Control , pp. 56-64, May 1959. 

» 

209. Westdal, J. A. S. and W. HT. Lehra. "Optimal Control for Systems 

which. Include Pure Delays," Int 1 ! . JN Control , Vol. 11, No. 4, 
pp. 599-610, April 1970. See also Correspondence Int 1 1 . ^J. 
Control , Vol. 13, No. 1, pp. 201-203. 

210* Wt.erzbickl, A. "Maximum Principle for Processes with Non-Trivial 
Delay of the Control," Automation and Remote Control ; 
pp. 1543-1549, October 1970. 

* 

211. Yuan-Xun, Qin, Liow long-Qing and Uang Lian. "Effect of Time Lags 

on Stability of Dynamical Systems," ’ Proc . 1st IFAC Congress 
(Moscow), pp. 26-32, 1960. 

212. Zhiyoglyadov, V. P. "Optimal Dual Control of Plants with Pure 

Delay," Automation and Remote Control, pp. 52-63, January 
1964. 

213. Zubov, W. ’£. "On the Theory* of Linear Stationary Systems with 

Lagging Argument," Reports from Institutes of Higher Educa¬ 
tion , Math ., No . 6 (7) , pp. 86-95^ (Russian), 1958. 



